You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2020/03/02 11:47:15 UTC

Build failed in Jenkins: beam_PostCommit_XVR_Spark #234

See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/234/display/redirect?page=changes>

Changes:

[iemejia] [BEAM-9342[ Update bytebuddy to version 1.10.8


------------------------------------------
[...truncated 691.26 KB...]
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f7f82909938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:30 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid_4\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey_12\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse_14\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>)_15\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode)_17\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault_18\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f7f82909938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f7f82909938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f7f82909938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f7f82909938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f7f82909938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 67.176s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED
> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 16m 22s
102 actionable tasks: 86 executed, 14 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/tpm4xsuryjkl6

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Jenkins build is back to normal : beam_PostCommit_XVR_Spark #256

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/256/display/redirect?page=changes>


---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #255

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/255/display/redirect>

Changes:


------------------------------------------
[...truncated 687.42 KB...]
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_globally (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fa092cc4cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:30 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid_4\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey_12\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse_14\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>)_15\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode)_17\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault_18\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fa092cc4cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fa092cc4cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fa092cc4cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fa092cc4cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:36 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Main1/Impulse_3\n  Main1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Main1/FlatMap(<lambda at core.py:2643>)_4\n  Main1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Main1/Map(decode)_6\n  Main1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Main2/Impulse_8\n  Main2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Main2/FlatMap(<lambda at core.py:2643>)_9\n  Main2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Main2/Map(decode)_11\n  Main2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Side/Impulse_13\n  Side/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Side/FlatMap(<lambda at core.py:2643>)_14\n  Side/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Side/Map(decode)_16\n  Side/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_8_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:multi)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:multi)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_8_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:multi)/Map(<lambda at expansion_service_test.py:115>)_4\n  ExternalTransform(beam:transforms:xlang:test:multi)/Map(<lambda at expansion_service_test.py:115>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_8_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:multi)/Map(<lambda at expansion_service_test.py:118>)_5\n  ExternalTransform(beam:transforms:xlang:test:multi)/Map(<lambda at expansion_service_test.py:118>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Create/Impulse_39\n  CheckSide/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Create/FlatMap(<lambda at core.py:2643>)_40\n  CheckSide/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Create/Map(decode)_42\n  CheckSide/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/WindowInto(WindowIntoFn)_43\n  CheckSide/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/ToVoidKey_44\n  CheckSide/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Group/pair_with_0_46\n  CheckSide/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Group/pair_with_1_47\n  CheckSide/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Group/Flatten_48\n  CheckSide/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Group/GroupByKey_49\n  CheckSide/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Group/Map(_merge_tagged_vals_under_key)_53\n  CheckSide/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Unkey_54\n  CheckSide/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Match_55\n  CheckSide/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fa092cc4cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:217>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:217>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fa092cc4cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 91.999s

OK

> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 3 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 18m 40s
102 actionable tasks: 78 executed, 22 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/gmhyvetd2cdcq

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #254

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/254/display/redirect>

Changes:


------------------------------------------
[...truncated 688.05 KB...]
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_globally (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f9aa5fdacf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:30 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid_4\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey_12\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse_14\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>)_15\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode)_17\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault_18\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f9aa5fdacf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f9aa5fdacf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f9aa5fdacf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f9aa5fdacf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:36 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Main1/Impulse_3\n  Main1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Main1/FlatMap(<lambda at core.py:2643>)_4\n  Main1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Main1/Map(decode)_6\n  Main1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Main2/Impulse_8\n  Main2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Main2/FlatMap(<lambda at core.py:2643>)_9\n  Main2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Main2/Map(decode)_11\n  Main2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Side/Impulse_13\n  Side/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Side/FlatMap(<lambda at core.py:2643>)_14\n  Side/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Side/Map(decode)_16\n  Side/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_8_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:multi)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:multi)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_8_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:multi)/Map(<lambda at expansion_service_test.py:115>)_4\n  ExternalTransform(beam:transforms:xlang:test:multi)/Map(<lambda at expansion_service_test.py:115>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_8_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:multi)/Map(<lambda at expansion_service_test.py:118>)_5\n  ExternalTransform(beam:transforms:xlang:test:multi)/Map(<lambda at expansion_service_test.py:118>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Create/Impulse_39\n  CheckSide/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Create/FlatMap(<lambda at core.py:2643>)_40\n  CheckSide/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Create/Map(decode)_42\n  CheckSide/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/WindowInto(WindowIntoFn)_43\n  CheckSide/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/ToVoidKey_44\n  CheckSide/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Group/pair_with_0_46\n  CheckSide/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Group/pair_with_1_47\n  CheckSide/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Group/Flatten_48\n  CheckSide/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Group/GroupByKey_49\n  CheckSide/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Group/Map(_merge_tagged_vals_under_key)_53\n  CheckSide/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Unkey_54\n  CheckSide/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Match_55\n  CheckSide/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f9aa5fdacf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:217>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:217>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f9aa5fdacf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 88.975s

OK

> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 3 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 17m 49s
102 actionable tasks: 79 executed, 21 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/bfzyhhcorivwg

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #253

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/253/display/redirect?page=changes>

Changes:

[heejong] [BEAM-9415] fix postcommit xvr tests


------------------------------------------
[...truncated 689.74 KB...]
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_globally (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f4482749cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:30 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid_4\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey_12\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse_14\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>)_15\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode)_17\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault_18\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f4482749cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f4482749cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f4482749cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f4482749cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:36 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Main1/Impulse_3\n  Main1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Main1/FlatMap(<lambda at core.py:2643>)_4\n  Main1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Main1/Map(decode)_6\n  Main1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Main2/Impulse_8\n  Main2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Main2/FlatMap(<lambda at core.py:2643>)_9\n  Main2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Main2/Map(decode)_11\n  Main2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Side/Impulse_13\n  Side/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Side/FlatMap(<lambda at core.py:2643>)_14\n  Side/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Side/Map(decode)_16\n  Side/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_8_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:multi)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:multi)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_8_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:multi)/Map(<lambda at expansion_service_test.py:115>)_4\n  ExternalTransform(beam:transforms:xlang:test:multi)/Map(<lambda at expansion_service_test.py:115>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_8_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:multi)/Map(<lambda at expansion_service_test.py:118>)_5\n  ExternalTransform(beam:transforms:xlang:test:multi)/Map(<lambda at expansion_service_test.py:118>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Create/Impulse_39\n  CheckSide/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Create/FlatMap(<lambda at core.py:2643>)_40\n  CheckSide/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Create/Map(decode)_42\n  CheckSide/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/WindowInto(WindowIntoFn)_43\n  CheckSide/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/ToVoidKey_44\n  CheckSide/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Group/pair_with_0_46\n  CheckSide/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Group/pair_with_1_47\n  CheckSide/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Group/Flatten_48\n  CheckSide/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Group/GroupByKey_49\n  CheckSide/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Group/Map(_merge_tagged_vals_under_key)_53\n  CheckSide/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Unkey_54\n  CheckSide/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CheckSide/Match_55\n  CheckSide/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f4482749cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:217>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:217>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f4482749cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 83.710s

OK

> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 3 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 17m 43s
102 actionable tasks: 89 executed, 11 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/pu2nnggsjl6ka

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #252

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/252/display/redirect?page=changes>

Changes:

[github] [BEAM-9413] fix beam_PostCommit_Py_ValCon (#11023)


------------------------------------------
[...truncated 699.70 KB...]
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fe9d4d46cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fe9d4d46cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fe9d4d46cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fe9d4d46cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fe9d4d46cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 99.869s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED

> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
Skipping invalid pid: 22457.
Stopping expansion service pid: 22458.

> Task :runners:spark:job-server:sparkJobServerCleanup
Stopping job server pid: 11374.

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 22m 32s
102 actionable tasks: 78 executed, 22 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/sy4vwf2qefjmu

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #251

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/251/display/redirect?page=changes>

Changes:

[iemejia] [website] Update link to environment_type (SDK harness configuration)

[iemejia] Fix typo on python code

[rohde.samuel] [BEAM-8335] Add PCollection to Dataframe logic for InteractiveRunner.

[github] [BEAM-8575] Modified trigger test to work for different runners.


------------------------------------------
[...truncated 690.08 KB...]
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fb0e8413cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:30 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid_4\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey_12\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse_14\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>)_15\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode)_17\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault_18\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fb0e8413cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fb0e8413cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fb0e8413cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fb0e8413cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fb0e8413cf8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 84.698s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED
> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 17m 10s
102 actionable tasks: 78 executed, 22 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/7ryipuclah372

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #250

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/250/display/redirect?page=changes>

Changes:

[chadrik] [BEAM-7746] Runtime change to timestamp/duration equality


------------------------------------------
[...truncated 719.05 KB...]
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f321947eaa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
ERROR
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f321947eaa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
ERROR

======================================================================
ERROR: test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 168, in test_flatten
    assert_that(res, equal_to([1, 2, 3, 4, 5, 6]))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 522, in __exit__
    self.run().wait_until_finish()
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 114, in run
    state = result.wait_until_finish()
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 522, in wait_until_finish
    for state_response in self._state_stream:
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/grpc/_channel.py",> line 413, in next
    return self._next()
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/grpc/_channel.py",> line 706, in _next
    raise self
_MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1583264853.108147681","description":"Error received from peer ipv4:127.0.0.1:18091","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Socket closed","grpc_status":14}"
>
-------------------- >> begin captured logging << --------------------
root: WARNING: Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
root: INFO: Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
apache_beam.runners.portability.fn_api_runner_transforms: INFO: ==================== <function lift_combiners at 0x7f321947eaa0> ====================
apache_beam.runners.portability.fn_api_runner_transforms: DEBUG: 29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner_transforms: DEBUG: Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'dataflow_kms_key' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'enable_streaming_engine' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'project' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_region' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_name' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'runner' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'temp_location' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'streaming' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'experiments' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_cache_millis' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_endpoint' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'output_executable_path' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'sdk_worker_parallelism' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'files_to_stage' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'flink_master' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'pubsub_root_url' was already added
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to RUNNING
--------------------- >> end captured logging << ---------------------

======================================================================
ERROR: test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 104, in test_group_by_key
    assert_that(res, equal_to(['0:1,2', '1:3']))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 522, in __exit__
    self.run().wait_until_finish()
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
    False if self.not_use_test_runner_api else test_runner_api))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 508, in run
    return self.runner.run_pipeline(self, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 386, in run_pipeline
    job_service_handle = self.create_job_service(options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 293, in create_job_service
    return JobServiceHandle(server.start(), options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/portability/job_server.py",> line 59, in start
    grpc.channel_ready_future(channel).result(timeout=self._timeout)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/grpc/_utilities.py",> line 140, in result
    self._block(timeout)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/grpc/_utilities.py",> line 86, in _block
    raise grpc.FutureTimeoutError()
FutureTimeoutError: 
-------------------- >> begin captured logging << --------------------
root: WARNING: Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
root: INFO: Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
apache_beam.runners.portability.fn_api_runner_transforms: INFO: ==================== <function lift_combiners at 0x7f321947eaa0> ====================
apache_beam.runners.portability.fn_api_runner_transforms: DEBUG: 17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner_transforms: DEBUG: Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
--------------------- >> end captured logging << ---------------------

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


======================================================================
ERROR: test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 185, in test_partition
    assert_that(res['1'], equal_to([1, 3, 5]), label='check_odd')
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 522, in __exit__
    self.run().wait_until_finish()
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
    False if self.not_use_test_runner_api else test_runner_api))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 508, in run
    return self.runner.run_pipeline(self, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 386, in run_pipeline
    job_service_handle = self.create_job_service(options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 293, in create_job_service
    return JobServiceHandle(server.start(), options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/portability/job_server.py",> line 59, in start
    grpc.channel_ready_future(channel).result(timeout=self._timeout)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/grpc/_utilities.py",> line 140, in result
    self._block(timeout)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/grpc/_utilities.py",> line 86, in _block
    raise grpc.FutureTimeoutError()
FutureTimeoutError: 
-------------------- >> begin captured logging << --------------------
root: WARNING: Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
root: INFO: Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
apache_beam.runners.portability.fn_api_runner_transforms: INFO: ==================== <function lift_combiners at 0x7f321947eaa0> ====================
apache_beam.runners.portability.fn_api_runner_transforms: DEBUG: 35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner_transforms: DEBUG: Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
--------------------- >> end captured logging << ---------------------

======================================================================
ERROR: test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 69, in test_prefix
    assert_that(res, equal_to(['0a', '0b']))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 522, in __exit__
    self.run().wait_until_finish()
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
    False if self.not_use_test_runner_api else test_runner_api))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 508, in run
    return self.runner.run_pipeline(self, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 386, in run_pipeline
    job_service_handle = self.create_job_service(options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 293, in create_job_service
    return JobServiceHandle(server.start(), options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/portability/job_server.py",> line 59, in start
    grpc.channel_ready_future(channel).result(timeout=self._timeout)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/grpc/_utilities.py",> line 140, in result
    self._block(timeout)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/grpc/_utilities.py",> line 86, in _block
    raise grpc.FutureTimeoutError()
FutureTimeoutError: 
-------------------- >> begin captured logging << --------------------
root: WARNING: Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
root: INFO: Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
apache_beam.runners.portability.fn_api_runner_transforms: INFO: ==================== <function lift_combiners at 0x7f321947eaa0> ====================
apache_beam.runners.portability.fn_api_runner_transforms: DEBUG: 21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner_transforms: DEBUG: Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 403.417s

FAILED (errors=5)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED
> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 22m 54s
102 actionable tasks: 78 executed, 22 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/uwxiqmzfr5aoa

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #249

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/249/display/redirect?page=changes>

Changes:

[lcwik] [BEAM-9288] Bump version number vendored gRPC build.


------------------------------------------
[...truncated 688.76 KB...]
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fe9935c3aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:30 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid_4\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey_12\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse_14\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>)_15\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode)_17\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault_18\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fe9935c3aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fe9935c3aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fe9935c3aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fe9935c3aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fe9935c3aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 81.325s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED
> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 16m 30s
102 actionable tasks: 78 executed, 22 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/m4bu2sgm7p27k

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #248

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/248/display/redirect>

Changes:


------------------------------------------
[...truncated 688.93 KB...]
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7feb6160caa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:30 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid_4\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey_12\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse_14\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>)_15\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode)_17\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault_18\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7feb6160caa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7feb6160caa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7feb6160caa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7feb6160caa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7feb6160caa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 65.033s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED
> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 16m 58s
102 actionable tasks: 78 executed, 22 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/4hojd5klavut6

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #247

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/247/display/redirect?page=changes>

Changes:

[fernandodiaz] [BEAM-9424] Allow grouping by LogicalType

[echauchot] Add metrics export to documentation on the website.

[github] [BEAM-8382] Add rate limit policy to KinesisIO.Read (#9765)


------------------------------------------
[...truncated 697.64 KB...]
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'flink_master' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'pubsub_root_url' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f5e1c898aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'flink_master' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'pubsub_root_url' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f5e1c898aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'flink_master' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'pubsub_root_url' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f5e1c898aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'flink_master' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'pubsub_root_url' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f5e1c898aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'flink_master' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'pubsub_root_url' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f5e1c898aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'flink_master' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'pubsub_root_url' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 137.295s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED
> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 17m 25s
102 actionable tasks: 78 executed, 22 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/rgbvzioo2sdso

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #246

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/246/display/redirect?page=changes>

Changes:

[kamil.wasilewski] Add integration test for AnnotateImage transform

[kamil.wasilewski] Fix: skip test if GCP dependencies are not installed


------------------------------------------
[...truncated 700.09 KB...]
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f83e6f37aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f83e6f37aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f83e6f37aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f83e6f37aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f83e6f37aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 69.640s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED

> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
Skipping invalid pid: 12968.
Stopping expansion service pid: 12969.

> Task :runners:spark:job-server:sparkJobServerCleanup
Stopping job server pid: 9179.

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 17m 20s
102 actionable tasks: 81 executed, 19 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/l6olhisvjuvsq

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #245

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/245/display/redirect?page=changes>

Changes:

[github] Revert "[BEAM-8335] Add PCollection to DataFrame logic for


------------------------------------------
[...truncated 689.08 KB...]
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fb4cb6029b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:30 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid_4\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey_12\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse_14\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>)_15\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode)_17\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault_18\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fb4cb6029b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fb4cb6029b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fb4cb6029b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fb4cb6029b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fb4cb6029b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 67.506s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED
> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 16m 31s
102 actionable tasks: 78 executed, 22 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/2zbec2a3etxhc

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #244

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/244/display/redirect>

Changes:


------------------------------------------
[...truncated 691.18 KB...]
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f762860ac08> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:30 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid_4\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey_12\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse_14\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>)_15\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode)_17\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault_18\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f762860ac08> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f762860ac08> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f762860ac08> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f762860ac08> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f762860ac08> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 69.685s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED
> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 16m 23s
102 actionable tasks: 82 executed, 18 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/37xyjhdctq3ng

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #243

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/243/display/redirect>

Changes:


------------------------------------------
[...truncated 76.24 KB...]
Collecting rsa>=3.1.4
  Using cached rsa-4.0-py2.py3-none-any.whl (38 kB)
Collecting pyasn1>=0.1.7
  Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting pyasn1-modules>=0.0.5
  Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: setuptools in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from protobuf<4,>=3.5.0.post1->apache-beam==2.21.0.dev0) (44.0.0)
Collecting pyparsing>=2.1.4
  Using cached pyparsing-2.4.6-py2.py3-none-any.whl (67 kB)
Collecting fasteners>=0.14
  Using cached fasteners-0.15-py2.py3-none-any.whl (23 kB)
Collecting google-api-core[grpc]<2.0.0dev,>=1.6.0
  Using cached google_api_core-1.16.0-py2.py3-none-any.whl (70 kB)
Processing /home/jenkins/.cache/pip/wheels/de/3a/83/77a1e18e1a8757186df834b86ce6800120ac9c79cd8ca4091b/grpc_google_iam_v1-0.12.3-cp27-none-any.whl
Collecting google-auth<2.0dev,>=1.9.0
  Using cached google_auth-1.11.2-py2.py3-none-any.whl (76 kB)
Collecting google-resumable-media<0.6dev,>=0.5.0
  Using cached google_resumable_media-0.5.0-py2.py3-none-any.whl (38 kB)
Processing /home/jenkins/.cache/pip/wheels/2c/f9/7f/6eb87e636072bf467e25348bbeb96849333e6a080dca78f706/googleapis_common_protos-1.51.0-cp27-none-any.whl
Collecting monotonic>=0.6; python_version == "2.7"
  Using cached monotonic-1.5-py2.py3-none-any.whl (5.3 kB)
Collecting atomicwrites>=1.0
  Using cached atomicwrites-1.3.0-py2.py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (0.13.1)
Collecting packaging
  Using cached packaging-20.1-py2.py3-none-any.whl (36 kB)
Collecting attrs>=17.4.0
  Using cached attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Requirement already satisfied: importlib-metadata>=0.12; python_version < "3.8" in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (1.5.0)
Collecting wcwidth
  Using cached wcwidth-0.1.8-py2.py3-none-any.whl (17 kB)
Collecting more-itertools<6.0.0,>=4.0.0; python_version <= "2.7"
  Using cached more_itertools-5.0.0-py2-none-any.whl (52 kB)
Requirement already satisfied: pathlib2>=2.2.0; python_version < "3.6" in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (2.3.5)
Requirement already satisfied: py>=1.5.0 in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (1.8.1)
Collecting pytest-forked
  Using cached pytest_forked-1.1.3-py2.py3-none-any.whl (4.5 kB)
Collecting execnet>=1.1
  Using cached execnet-1.7.1-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2019.11.28-py2.py3-none-any.whl (156 kB)
Collecting idna<3,>=2.5
  Using cached idna-2.9-py2.py3-none-any.whl (58 kB)
Collecting chardet<4,>=3.0.2
  Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
  Using cached urllib3-1.25.8-py2.py3-none-any.whl (125 kB)
Requirement already satisfied: contextlib2; python_version < "3" in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from importlib-metadata>=0.12; python_version < "3.8"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (0.6.0.post1)
Requirement already satisfied: zipp>=0.5 in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from importlib-metadata>=0.12; python_version < "3.8"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (1.2.0)
Requirement already satisfied: configparser>=3.5; python_version < "3" in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from importlib-metadata>=0.12; python_version < "3.8"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (4.0.2)
Requirement already satisfied: scandir; python_version < "3.5" in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from pathlib2>=2.2.0; python_version < "3.6"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (1.10.0)
Collecting apipkg>=1.4
  Using cached apipkg-1.5-py2.py3-none-any.whl (4.9 kB)
Building wheels for collected packages: apache-beam
  Building wheel for apache-beam (setup.py): started
  Building wheel for apache-beam (setup.py): finished with status 'done'
  Created wheel for apache-beam: filename=apache_beam-2.21.0.dev0-py2-none-any.whl size=1949567 sha256=269681dadd0621df61c7dda87618264d4ed425cc464b0bf407ebcbbdd9a48025
  Stored in directory: /home/jenkins/.cache/pip/wheels/f2/b0/4f/c4389c81ddb5fa6a9261073a77dcdd1bd828e0d4b43417b55c
Successfully built apache-beam
Installing collected packages: crcmod, dill, fastavro, docopt, certifi, idna, chardet, urllib3, requests, hdfs, httplib2, pbr, funcsigs, mock, numpy, pymongo, pyasn1, rsa, pyasn1-modules, oauth2client, pyparsing, pydot, python-dateutil, pytz, avro, pyvcf, pyarrow, typing-extensions, cachetools, monotonic, fasteners, google-apitools, google-auth, googleapis-common-protos, google-api-core, google-cloud-core, google-cloud-datastore, grpc-google-iam-v1, google-cloud-pubsub, google-resumable-media, google-cloud-bigquery, google-cloud-bigtable, google-cloud-spanner, grpcio-gcp, google-cloud-dlp, google-cloud-language, google-cloud-videointelligence, google-cloud-vision, proto-google-cloud-datastore-v1, googledatastore, freezegun, nose, nose-xunitmp, pandas, parameterized, pyhamcrest, pyyaml, requests-mock, tenacity, atomicwrites, packaging, attrs, wcwidth, more-itertools, pytest, pytest-forked, apipkg, execnet, pytest-xdist, pytest-timeout, apache-beam
Successfully installed apache-beam-2.21.0.dev0 apipkg-1.5 atomicwrites-1.3.0 attrs-19.3.0 avro-1.9.2 cachetools-3.1.1 certifi-2019.11.28 chardet-3.0.4 crcmod-1.7 dill-0.3.1.1 docopt-0.6.2 execnet-1.7.1 fastavro-0.21.24 fasteners-0.15 freezegun-0.3.15 funcsigs-1.0.2 google-api-core-1.16.0 google-apitools-0.5.28 google-auth-1.11.2 google-cloud-bigquery-1.24.0 google-cloud-bigtable-1.0.0 google-cloud-core-1.3.0 google-cloud-datastore-1.7.4 google-cloud-dlp-0.13.0 google-cloud-language-1.3.0 google-cloud-pubsub-1.0.2 google-cloud-spanner-1.13.0 google-cloud-videointelligence-1.13.0 google-cloud-vision-0.42.0 google-resumable-media-0.5.0 googleapis-common-protos-1.51.0 googledatastore-7.0.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 hdfs-2.5.8 httplib2-0.12.0 idna-2.9 mock-2.0.0 monotonic-1.5 more-itertools-5.0.0 nose-1.3.7 nose-xunitmp-0.4.1 numpy-1.16.6 oauth2client-3.0.0 packaging-20.1 pandas-0.24.2 parameterized-0.7.1 pbr-5.4.4 proto-google-cloud-datastore-v1-0.90.4 pyarrow-0.15.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pydot-1.4.1 pyhamcrest-1.10.1 pymongo-3.10.1 pyparsing-2.4.6 pytest-4.6.9 pytest-forked-1.1.3 pytest-timeout-1.3.4 pytest-xdist-1.31.0 python-dateutil-2.8.1 pytz-2019.3 pyvcf-0.6.8 pyyaml-5.3 requests-2.23.0 requests-mock-1.7.0 rsa-4.0 tenacity-5.1.5 typing-extensions-3.7.4.1 urllib3-1.25.8 wcwidth-0.1.8

> Task :runners:spark:job-server:shadowJar

> Task :runners:spark:job-server:sparkJobServerSetup
Launching job server @ 18091 ...

> Task :sdks:go:resolveBuildDependencies
Resolving google.golang.org/api: commit='386d4e5f4f92f86e6aec85985761bba4b938a2d5', urls=[https://code.googlesource.com/google-api-go-client]
Resolving google.golang.org/genproto: commit='2b5a72b8730b0b16380010cfe5286c42108d88e7', urls=[https://github.com/google/go-genproto]
Resolving google.golang.org/grpc: commit='7646b5360d049a7ca31e9133315db43456f39e2e', urls=[https://github.com/grpc/grpc-go]
Resolving cached github.com/coreos/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/coreos/etcd.git, git@github.com:coreos/etcd.git]
Resolving cached github.com/coreos/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/coreos/etcd.git, git@github.com:coreos/etcd.git]

> Task :sdks:go:installDependencies
> Task :sdks:go:buildLinuxAmd64
> Task :sdks:go:goBuild

> Task :sdks:python:container:resolveBuildDependencies
Resolving ./github.com/apache/beam/sdks/go@<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/go>

> Task :sdks:java:container:resolveBuildDependencies
Resolving ./github.com/apache/beam/sdks/go@<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/go>

> Task :sdks:java:container:installDependencies
> Task :sdks:python:container:installDependencies
> Task :sdks:java:container:buildLinuxAmd64
> Task :sdks:java:container:goBuild
> Task :sdks:java:container:dockerPrepare
> Task :sdks:python:container:buildDarwinAmd64
> Task :sdks:python:container:buildLinuxAmd64
> Task :sdks:python:container:goBuild

> Task :sdks:java:container:docker
Sending build context to Docker daemon  127.5MB
Step 1/11 : FROM openjdk:8
 ---> cdf26cc71b50
Step 2/11 : MAINTAINER "Apache Beam <de...@beam.apache.org>"

> Task :sdks:python:container:py2:copyLauncherDependencies
> Task :sdks:python:container:py2:dockerPrepare

> Task :sdks:python:container:py2:docker
Fetched 7937 kB in 2s (3267 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
libyaml-dev is already the newest version (0.1.7-2).
Suggested packages:
  distcc
The following NEW packages will be installed:
  ccache libsnappy-dev libsnappy1v5
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 253 kB of archives.
After this operation, 565 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian stretch/main amd64 ccache amd64 3.3.4-1 [136 kB]
Get:2 http://deb.debian.org/debian stretch/main amd64 libsnappy1v5 amd64 1.1.3-3 [51.0 kB]
Get:3 http://deb.debian.org/debian stretch/main amd64 libsnappy-dev amd64 1.1.3-3 [65.8 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 253 kB in 0s (3697 kB/s)
Selecting previously unselected package ccache.
(Reading database ... 
(Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%(Reading database ... 30535 files and directories currently installed.)
Preparing to unpack .../ccache_3.3.4-1_amd64.deb ...
Unpacking ccache (3.3.4-1) ...
Selecting previously unselected package libsnappy1v5:amd64.
Preparing to unpack .../libsnappy1v5_1.1.3-3_amd64.deb ...
Unpacking libsnappy1v5:amd64 (1.1.3-3) ...
Selecting previously unselected package libsnappy-dev:amd64.
Preparing to unpack .../libsnappy-dev_1.1.3-3_amd64.deb ...
Unpacking libsnappy-dev:amd64 (1.1.3-3) ...
Setting up ccache (3.3.4-1) ...
Updating symlinks in /usr/lib/ccache ...
Processing triggers for libc-bin (2.24-11+deb9u4) ...
Setting up libsnappy1v5:amd64 (1.1.3-3) ...
Setting up libsnappy-dev:amd64 (1.1.3-3) ...
Processing triggers for libc-bin (2.24-11+deb9u4) ...
Removing intermediate container e043cc20c448
 ---> c73bb04873ed
Step 5/15 : COPY target/base_image_requirements.txt /tmp/base_image_requirements.txt
 ---> 4a8e520699bd
Step 6/15 : RUN     pip install -r /tmp/base_image_requirements.txt &&     python -c "from google.protobuf.internal import api_implementation; assert api_implementation._default_implementation_type == 'cpp'; print ('Verified fast protobuf used.')" &&     rm -rf /root/.cache/pip
 ---> Running in 75fe793cc87c
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Ignoring avro-python3: markers 'python_version >= "3.4"' don't match your environment
Collecting avro==1.8.2
  Downloading avro-1.8.2.tar.gz (43 kB)
Collecting fastavro==0.21.24
  Downloading fastavro-0.21.24-cp27-cp27mu-manylinux1_x86_64.whl (1.0 MB)
Collecting crcmod==1.7
  Downloading crcmod-1.7.tar.gz (89 kB)
Collecting dill==0.3.0
  Downloading dill-0.3.0.tar.gz (151 kB)
Collecting future==0.17.1
  Downloading future-0.17.1.tar.gz (829 kB)
Collecting futures==3.2.0
  Downloading futures-3.2.0-py2-none-any.whl (15 kB)
Collecting grpcio==1.22.0
  Downloading grpcio-1.22.0-cp27-cp27mu-manylinux1_x86_64.whl (2.2 MB)
Collecting hdfs==2.5.6
  Downloading hdfs-2.5.6.tar.gz (41 kB)
Collecting httplib2==0.12.0
  Downloading httplib2-0.12.0.tar.gz (218 kB)
Collecting mock==2.0.0
  Downloading mock-2.0.0-py2.py3-none-any.whl (56 kB)
Collecting oauth2client==3.0.0
  Downloading oauth2client-3.0.0.tar.gz (77 kB)
Collecting protobuf==3.9.0
  Downloading protobuf-3.9.0-cp27-cp27mu-manylinux1_x86_64.whl (1.2 MB)
Collecting pyarrow==0.13.0
  Downloading pyarrow-0.13.0-cp27-cp27mu-manylinux1_x86_64.whl (14.3 MB)
Collecting pydot==1.4.1
  Downloading pydot-1.4.1-py2.py3-none-any.whl (19 kB)
Collecting pytz==2019.1
  Downloading pytz-2019.1-py2.py3-none-any.whl (510 kB)
Collecting pyvcf==0.6.8
  Downloading PyVCF-0.6.8.tar.gz (34 kB)
Collecting pyyaml==5.1
  Downloading PyYAML-5.1.tar.gz (274 kB)
Collecting typing==3.6.6
  Downloading typing-3.6.6-py2-none-any.whl (23 kB)
Collecting nose==1.3.7
  Downloading nose-1.3.7-py2-none-any.whl (154 kB)
Collecting google-apitools==0.5.28
  Downloading google_apitools-0.5.28-py2-none-any.whl (134 kB)
Collecting googledatastore==7.0.2
  Downloading googledatastore-7.0.2.tar.gz (13 kB)
Collecting google-cloud-pubsub==0.39.1
  Downloading google_cloud_pubsub-0.39.1-py2.py3-none-any.whl (99 kB)
Collecting google-cloud-bigquery==1.24.0
  Downloading google_cloud_bigquery-1.24.0-py2.py3-none-any.whl (165 kB)
Collecting proto-google-cloud-datastore-v1==0.90.4
  Downloading proto-google-cloud-datastore-v1-0.90.4.tar.gz (20 kB)
Collecting google-cloud-bigtable==0.32.1
  Downloading google_cloud_bigtable-0.32.1-py2.py3-none-any.whl (156 kB)
Collecting google-cloud-core==1.0.2
  Downloading google_cloud_core-1.0.2-py2.py3-none-any.whl (26 kB)
Collecting google-cloud-datastore==1.7.4
  Downloading google_cloud_datastore-1.7.4-py2.py3-none-any.whl (82 kB)
Collecting cython==0.29.10
  Downloading Cython-0.29.10-cp27-cp27mu-manylinux1_x86_64.whl (2.0 MB)
Collecting python-snappy==0.5.4
  Downloading python-snappy-0.5.4.tar.gz (21 kB)
Collecting numpy==1.16.4
  Downloading numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl (17.0 MB)
Collecting scipy==1.2.2
  Downloading scipy-1.2.2-cp27-cp27mu-manylinux1_x86_64.whl (24.8 MB)
Collecting pandas==0.24.2
  Downloading pandas-0.24.2-cp27-cp27mu-manylinux1_x86_64.whl (10.1 MB)
ERROR: Could not install packages due to an EnvironmentError: [Errno 28] No space left on device

The command '/bin/sh -c pip install -r /tmp/base_image_requirements.txt &&     python -c "from google.protobuf.internal import api_implementation; assert api_implementation._default_implementation_type == 'cpp'; print ('Verified fast protobuf used.')" &&     rm -rf /root/.cache/pip' returned a non-zero code: 1

> Task :sdks:python:container:py2:docker FAILED

> Task :runners:spark:job-server:sparkJobServerCleanup
Stopping job server pid: 29495.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:python:container:py2:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3m 5s
96 actionable tasks: 72 executed, 22 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/27deixx3ej2ae

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #242

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/242/display/redirect?page=changes>

Changes:

[github] [BEAM-2939] Java UnboundedSource SDF wrapper (#10897)


------------------------------------------
[...truncated 691.49 KB...]
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f5daca14c08> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:30 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid_4\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey_12\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse_14\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>)_15\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode)_17\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault_18\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f5daca14c08> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f5daca14c08> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f5daca14c08> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f5daca14c08> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f5daca14c08> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 64.717s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED
> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 15m 17s
102 actionable tasks: 84 executed, 16 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/uyruz75fnz23k

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #241

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/241/display/redirect?page=changes>

Changes:

[github] [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner.


------------------------------------------
[...truncated 76.55 KB...]
  Using cached google_api_core-1.16.0-py2.py3-none-any.whl (70 kB)
Processing /home/jenkins/.cache/pip/wheels/de/3a/83/77a1e18e1a8757186df834b86ce6800120ac9c79cd8ca4091b/grpc_google_iam_v1-0.12.3-cp27-none-any.whl
Collecting google-auth<2.0dev,>=1.9.0
  Using cached google_auth-1.11.2-py2.py3-none-any.whl (76 kB)
Collecting google-resumable-media<0.6dev,>=0.5.0
  Using cached google_resumable_media-0.5.0-py2.py3-none-any.whl (38 kB)
Processing /home/jenkins/.cache/pip/wheels/2c/f9/7f/6eb87e636072bf467e25348bbeb96849333e6a080dca78f706/googleapis_common_protos-1.51.0-cp27-none-any.whl
Collecting monotonic>=0.6; python_version == "2.7"
  Using cached monotonic-1.5-py2.py3-none-any.whl (5.3 kB)
Collecting atomicwrites>=1.0
  Using cached atomicwrites-1.3.0-py2.py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (0.13.1)
Collecting packaging
  Using cached packaging-20.1-py2.py3-none-any.whl (36 kB)
Collecting attrs>=17.4.0
  Using cached attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Requirement already satisfied: importlib-metadata>=0.12; python_version < "3.8" in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (1.5.0)
Collecting wcwidth
  Using cached wcwidth-0.1.8-py2.py3-none-any.whl (17 kB)
Collecting more-itertools<6.0.0,>=4.0.0; python_version <= "2.7"
  Using cached more_itertools-5.0.0-py2-none-any.whl (52 kB)
Requirement already satisfied: pathlib2>=2.2.0; python_version < "3.6" in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (2.3.5)
Requirement already satisfied: py>=1.5.0 in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (1.8.1)
Collecting pytest-forked
  Using cached pytest_forked-1.1.3-py2.py3-none-any.whl (4.5 kB)
Collecting execnet>=1.1
  Using cached execnet-1.7.1-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2019.11.28-py2.py3-none-any.whl (156 kB)
Collecting idna<3,>=2.5
  Using cached idna-2.9-py2.py3-none-any.whl (58 kB)
Collecting chardet<4,>=3.0.2
  Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
  Using cached urllib3-1.25.8-py2.py3-none-any.whl (125 kB)
Requirement already satisfied: contextlib2; python_version < "3" in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from importlib-metadata>=0.12; python_version < "3.8"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (0.6.0.post1)
Requirement already satisfied: zipp>=0.5 in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from importlib-metadata>=0.12; python_version < "3.8"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (1.2.0)
Requirement already satisfied: configparser>=3.5; python_version < "3" in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from importlib-metadata>=0.12; python_version < "3.8"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (4.0.2)
Requirement already satisfied: scandir; python_version < "3.5" in <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from pathlib2>=2.2.0; python_version < "3.6"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (1.10.0)
Collecting apipkg>=1.4
  Using cached apipkg-1.5-py2.py3-none-any.whl (4.9 kB)
Building wheels for collected packages: apache-beam
  Building wheel for apache-beam (setup.py): started
  Building wheel for apache-beam (setup.py): finished with status 'done'
  Created wheel for apache-beam: filename=apache_beam-2.21.0.dev0-py2-none-any.whl size=1949567 sha256=d25f1cd264b55eb87ff5fb2c3cfaa2aa218cf9113946bba0f877a6f1e2361ed2
  Stored in directory: /home/jenkins/.cache/pip/wheels/f2/b0/4f/c4389c81ddb5fa6a9261073a77dcdd1bd828e0d4b43417b55c
Successfully built apache-beam
Installing collected packages: crcmod, dill, fastavro, docopt, certifi, idna, chardet, urllib3, requests, hdfs, httplib2, pbr, funcsigs, mock, numpy, pymongo, pyasn1, rsa, pyasn1-modules, oauth2client, pyparsing, pydot, python-dateutil, pytz, avro, pyvcf, pyarrow, typing-extensions, cachetools, monotonic, fasteners, google-apitools, google-auth, googleapis-common-protos, google-api-core, google-cloud-core, google-cloud-datastore, grpc-google-iam-v1, google-cloud-pubsub, google-resumable-media, google-cloud-bigquery, google-cloud-bigtable, google-cloud-spanner, grpcio-gcp, google-cloud-dlp, google-cloud-language, google-cloud-videointelligence, google-cloud-vision, proto-google-cloud-datastore-v1, googledatastore, freezegun, nose, nose-xunitmp, pandas, parameterized, pyhamcrest, pyyaml, requests-mock, tenacity, atomicwrites, packaging, attrs, wcwidth, more-itertools, pytest, pytest-forked, apipkg, execnet, pytest-xdist, pytest-timeout, apache-beam
Successfully installed apache-beam-2.21.0.dev0 apipkg-1.5 atomicwrites-1.3.0 attrs-19.3.0 avro-1.9.2 cachetools-3.1.1 certifi-2019.11.28 chardet-3.0.4 crcmod-1.7 dill-0.3.1.1 docopt-0.6.2 execnet-1.7.1 fastavro-0.21.24 fasteners-0.15 freezegun-0.3.15 funcsigs-1.0.2 google-api-core-1.16.0 google-apitools-0.5.28 google-auth-1.11.2 google-cloud-bigquery-1.24.0 google-cloud-bigtable-1.0.0 google-cloud-core-1.3.0 google-cloud-datastore-1.7.4 google-cloud-dlp-0.13.0 google-cloud-language-1.3.0 google-cloud-pubsub-1.0.2 google-cloud-spanner-1.13.0 google-cloud-videointelligence-1.13.0 google-cloud-vision-0.42.0 google-resumable-media-0.5.0 googleapis-common-protos-1.51.0 googledatastore-7.0.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 hdfs-2.5.8 httplib2-0.12.0 idna-2.9 mock-2.0.0 monotonic-1.5 more-itertools-5.0.0 nose-1.3.7 nose-xunitmp-0.4.1 numpy-1.16.6 oauth2client-3.0.0 packaging-20.1 pandas-0.24.2 parameterized-0.7.1 pbr-5.4.4 proto-google-cloud-datastore-v1-0.90.4 pyarrow-0.15.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pydot-1.4.1 pyhamcrest-1.10.1 pymongo-3.10.1 pyparsing-2.4.6 pytest-4.6.9 pytest-forked-1.1.3 pytest-timeout-1.3.4 pytest-xdist-1.31.0 python-dateutil-2.8.1 pytz-2019.3 pyvcf-0.6.8 pyyaml-5.3 requests-2.23.0 requests-mock-1.7.0 rsa-4.0 tenacity-5.1.5 typing-extensions-3.7.4.1 urllib3-1.25.8 wcwidth-0.1.8

> Task :runners:spark:job-server:shadowJar

> Task :runners:spark:job-server:sparkJobServerSetup
Launching job server @ 18091 ...

> Task :sdks:go:resolveBuildDependencies
Resolving google.golang.org/api: commit='386d4e5f4f92f86e6aec85985761bba4b938a2d5', urls=[https://code.googlesource.com/google-api-go-client]
Resolving google.golang.org/genproto: commit='2b5a72b8730b0b16380010cfe5286c42108d88e7', urls=[https://github.com/google/go-genproto]
Resolving google.golang.org/grpc: commit='7646b5360d049a7ca31e9133315db43456f39e2e', urls=[https://github.com/grpc/grpc-go]
Resolving cached github.com/coreos/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/coreos/etcd.git, git@github.com:coreos/etcd.git]
Resolving cached github.com/coreos/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/coreos/etcd.git, git@github.com:coreos/etcd.git]

> Task :sdks:go:installDependencies
> Task :sdks:go:buildLinuxAmd64
> Task :sdks:go:goBuild

> Task :sdks:python:container:resolveBuildDependencies
Resolving ./github.com/apache/beam/sdks/go@<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/go>

> Task :sdks:java:container:resolveBuildDependencies
Resolving ./github.com/apache/beam/sdks/go@<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/go>

> Task :sdks:python:container:installDependencies
> Task :sdks:java:container:installDependencies
> Task :sdks:python:container:buildDarwinAmd64
> Task :sdks:java:container:buildLinuxAmd64
> Task :sdks:java:container:goBuild
> Task :sdks:java:container:dockerPrepare
> Task :sdks:python:container:buildLinuxAmd64
> Task :sdks:python:container:goBuild
> Task :sdks:python:container:py2:copyLauncherDependencies
> Task :sdks:python:container:py2:dockerPrepare
> Task :sdks:java:container:docker

> Task :sdks:python:container:py2:docker
Ign:1 http://deb.debian.org/debian stretch InRelease
Get:2 http://deb.debian.org/debian stretch-updates InRelease [91.0 kB]
Get:3 http://security.debian.org/debian-security stretch/updates InRelease [94.3 kB]
Get:4 http://deb.debian.org/debian stretch Release [118 kB]
Get:5 http://deb.debian.org/debian stretch Release.gpg [2410 B]
Get:6 http://security.debian.org/debian-security stretch/updates/main amd64 Packages [520 kB]
Get:7 http://deb.debian.org/debian stretch-updates/main amd64 Packages [27.9 kB]
Get:8 http://deb.debian.org/debian stretch/main amd64 Packages [7083 kB]
Fetched 7937 kB in 3s (2382 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
libyaml-dev is already the newest version (0.1.7-2).
Suggested packages:
  distcc
The following NEW packages will be installed:
  ccache libsnappy-dev libsnappy1v5
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 253 kB of archives.
After this operation, 565 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian stretch/main amd64 ccache amd64 3.3.4-1 [136 kB]
Get:2 http://deb.debian.org/debian stretch/main amd64 libsnappy1v5 amd64 1.1.3-3 [51.0 kB]
Get:3 http://deb.debian.org/debian stretch/main amd64 libsnappy-dev amd64 1.1.3-3 [65.8 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 253 kB in 0s (3677 kB/s)
Selecting previously unselected package ccache.
(Reading database ... 
(Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%(Reading database ... 30535 files and directories currently installed.)
Preparing to unpack .../ccache_3.3.4-1_amd64.deb ...
Unpacking ccache (3.3.4-1) ...
Selecting previously unselected package libsnappy1v5:amd64.
Preparing to unpack .../libsnappy1v5_1.1.3-3_amd64.deb ...
Unpacking libsnappy1v5:amd64 (1.1.3-3) ...
Selecting previously unselected package libsnappy-dev:amd64.
Preparing to unpack .../libsnappy-dev_1.1.3-3_amd64.deb ...
Unpacking libsnappy-dev:amd64 (1.1.3-3) ...
Setting up ccache (3.3.4-1) ...
Updating symlinks in /usr/lib/ccache ...
Processing triggers for libc-bin (2.24-11+deb9u4) ...
Setting up libsnappy1v5:amd64 (1.1.3-3) ...
Setting up libsnappy-dev:amd64 (1.1.3-3) ...
Processing triggers for libc-bin (2.24-11+deb9u4) ...
Removing intermediate container 199bcdcc3272
 ---> 2700e1923bd6
Step 5/15 : COPY target/base_image_requirements.txt /tmp/base_image_requirements.txt
 ---> ff119136e158
Step 6/15 : RUN     pip install -r /tmp/base_image_requirements.txt &&     python -c "from google.protobuf.internal import api_implementation; assert api_implementation._default_implementation_type == 'cpp'; print ('Verified fast protobuf used.')" &&     rm -rf /root/.cache/pip
 ---> Running in 06b7b7d030f8
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Ignoring avro-python3: markers 'python_version >= "3.4"' don't match your environment
Collecting avro==1.8.2
  Downloading avro-1.8.2.tar.gz (43 kB)
Collecting fastavro==0.21.24
  Downloading fastavro-0.21.24-cp27-cp27mu-manylinux1_x86_64.whl (1.0 MB)
Collecting crcmod==1.7
  Downloading crcmod-1.7.tar.gz (89 kB)
Collecting dill==0.3.0
  Downloading dill-0.3.0.tar.gz (151 kB)
Collecting future==0.17.1
  Downloading future-0.17.1.tar.gz (829 kB)
Collecting futures==3.2.0
  Downloading futures-3.2.0-py2-none-any.whl (15 kB)
Collecting grpcio==1.22.0
  Downloading grpcio-1.22.0-cp27-cp27mu-manylinux1_x86_64.whl (2.2 MB)
Collecting hdfs==2.5.6
  Downloading hdfs-2.5.6.tar.gz (41 kB)
Collecting httplib2==0.12.0
  Downloading httplib2-0.12.0.tar.gz (218 kB)
Collecting mock==2.0.0
  Downloading mock-2.0.0-py2.py3-none-any.whl (56 kB)
Collecting oauth2client==3.0.0
  Downloading oauth2client-3.0.0.tar.gz (77 kB)
Collecting protobuf==3.9.0
  Downloading protobuf-3.9.0-cp27-cp27mu-manylinux1_x86_64.whl (1.2 MB)
Collecting pyarrow==0.13.0
  Downloading pyarrow-0.13.0-cp27-cp27mu-manylinux1_x86_64.whl (14.3 MB)
Collecting pydot==1.4.1
  Downloading pydot-1.4.1-py2.py3-none-any.whl (19 kB)
Collecting pytz==2019.1
  Downloading pytz-2019.1-py2.py3-none-any.whl (510 kB)
Collecting pyvcf==0.6.8
  Downloading PyVCF-0.6.8.tar.gz (34 kB)
Collecting pyyaml==5.1
  Downloading PyYAML-5.1.tar.gz (274 kB)
Collecting typing==3.6.6
  Downloading typing-3.6.6-py2-none-any.whl (23 kB)
Collecting nose==1.3.7
  Downloading nose-1.3.7-py2-none-any.whl (154 kB)
Collecting google-apitools==0.5.28
  Downloading google_apitools-0.5.28-py2-none-any.whl (134 kB)
Collecting googledatastore==7.0.2
  Downloading googledatastore-7.0.2.tar.gz (13 kB)
Collecting google-cloud-pubsub==0.39.1
  Downloading google_cloud_pubsub-0.39.1-py2.py3-none-any.whl (99 kB)
Collecting google-cloud-bigquery==1.24.0
  Downloading google_cloud_bigquery-1.24.0-py2.py3-none-any.whl (165 kB)
Collecting proto-google-cloud-datastore-v1==0.90.4
  Downloading proto-google-cloud-datastore-v1-0.90.4.tar.gz (20 kB)
Collecting google-cloud-bigtable==0.32.1
  Downloading google_cloud_bigtable-0.32.1-py2.py3-none-any.whl (156 kB)
Collecting google-cloud-core==1.0.2
  Downloading google_cloud_core-1.0.2-py2.py3-none-any.whl (26 kB)
Collecting google-cloud-datastore==1.7.4
  Downloading google_cloud_datastore-1.7.4-py2.py3-none-any.whl (82 kB)
Collecting cython==0.29.10
  Downloading Cython-0.29.10-cp27-cp27mu-manylinux1_x86_64.whl (2.0 MB)
Collecting python-snappy==0.5.4
  Downloading python-snappy-0.5.4.tar.gz (21 kB)
Collecting numpy==1.16.4
  Downloading numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl (17.0 MB)
Collecting scipy==1.2.2
  Downloading scipy-1.2.2-cp27-cp27mu-manylinux1_x86_64.whl (24.8 MB)
Collecting pandas==0.24.2
  Downloading pandas-0.24.2-cp27-cp27mu-manylinux1_x86_64.whl (10.1 MB)
Collecting protorpc==0.12.0
  Downloading protorpc-0.12.0-py2-none-any.whl (134 kB)
Collecting python-gflags==3.0.6
  Downloading python-gflags-3.0.6.tar.gz (49 kB)
Collecting tensorflow==2.1.0
  Downloading tensorflow-2.1.0-cp27-cp27mu-manylinux2010_x86_64.whl (421.8 MB)
Collecting pymongo==3.8.0
  Downloading pymongo-3.8.0-cp27-cp27mu-manylinux1_x86_64.whl (410 kB)
Collecting tenacity>=5.0.2
  Downloading tenacity-6.1.0-py2.py3-none-any.whl (24 kB)
Collecting pyhamcrest<2.0,>=1.9
  Downloading PyHamcrest-1.10.1.tar.gz (43 kB)
ERROR: Could not install packages due to an EnvironmentError: [Errno 28] No space left on device

The command '/bin/sh -c pip install -r /tmp/base_image_requirements.txt &&     python -c "from google.protobuf.internal import api_implementation; assert api_implementation._default_implementation_type == 'cpp'; print ('Verified fast protobuf used.')" &&     rm -rf /root/.cache/pip' returned a non-zero code: 1

> Task :sdks:python:container:py2:docker FAILED
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:python:container:py2:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 4m 13s
96 actionable tasks: 72 executed, 22 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/gm63v6cbz6pay

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #240

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/240/display/redirect?page=changes>

Changes:

[lcwik] [BEAM-9397] Pass all but output receiver parameters to start


------------------------------------------
[...truncated 690.85 KB...]
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f654720b9b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:30 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid_4\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey_12\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse_14\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>)_15\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode)_17\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault_18\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f654720b9b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f654720b9b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f654720b9b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f654720b9b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f654720b9b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 64.226s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED
> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 15m 53s
102 actionable tasks: 83 executed, 17 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/34gpbson2wzem

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #239

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/239/display/redirect?page=changes>

Changes:

[github] Merge pull request #10968 from [BEAM-9381] Adding display data to


------------------------------------------
[...truncated 688.66 KB...]
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f23ed3bd9b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:30 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid_4\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey_12\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse_14\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>)_15\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode)_17\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault_18\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f23ed3bd9b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f23ed3bd9b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f23ed3bd9b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f23ed3bd9b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f23ed3bd9b0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 67.526s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED
> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 17m 2s
102 actionable tasks: 78 executed, 22 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/wpjtdtheo5nom

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #238

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/238/display/redirect>

Changes:


------------------------------------------
[...truncated 689.75 KB...]
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fbe0d00b938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:30 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid_4\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey_12\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse_14\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>)_15\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode)_17\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault_18\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fbe0d00b938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fbe0d00b938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fbe0d00b938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fbe0d00b938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7fbe0d00b938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 81.669s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED
> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 16m 9s
102 actionable tasks: 79 executed, 21 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/et44gymisibfo

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #237

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/237/display/redirect?page=changes>

Changes:

[aromanenko.dev] [BEAM-8925] Tika version update to 1.23


------------------------------------------
[...truncated 700.10 KB...]
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f477388f938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f477388f938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f477388f938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f477388f938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f477388f938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 69.376s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED

> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
Skipping invalid pid: 18017.
Stopping expansion service pid: 18018.

> Task :runners:spark:job-server:sparkJobServerCleanup
Stopping job server pid: 11631.

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 16m 39s
102 actionable tasks: 84 executed, 16 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/n626sfmhbsdv6

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #236

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/236/display/redirect?page=changes>

Changes:

[12602502+Ardagan] [BEAM-8327] Override Gradle cache for community metrics prober


------------------------------------------
[...truncated 689.25 KB...]
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f26f8915938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:30 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid_4\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey_12\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse_14\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>)_15\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode)_17\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault_18\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f26f8915938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f26f8915938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f26f8915938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f26f8915938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f26f8915938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 79.400s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED
> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 17m 39s
102 actionable tasks: 78 executed, 22 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/2iiep7vlealna

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_XVR_Spark #235

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/235/display/redirect>

Changes:


------------------------------------------
[...truncated 690.28 KB...]
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f9ece5bb938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:30 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid_4\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/KeyWithVoid:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey_12\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/UnKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse_14\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>)_15\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode)_17\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/DoOnce/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_4_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault_18\n  ExternalTransform(beam:transforms:xlang:test:comgl)/CombineGlobally(sum)/InjectDefault:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_combine_per_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f9ece5bb938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:24 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Precombine:beam:transform:combine_per_key_precombine:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Group:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs\n  ExternalTransform(beam:transforms:xlang:test:compk)/CombinePerKey(sum)/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_flatten (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f9ece5bb938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_col1/Impulse_3\n  col1/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/FlatMap(<lambda at core.py:2643>)_4\n  col1/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  col1/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  col1/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  col1/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col1/Map(decode)_16\n  col1/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Impulse_18\n  col2/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/FlatMap(<lambda at core.py:2643>)_19\n  col2/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/AddRandomKeys_22\n  col2/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_25\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n  col2/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys_30\n  col2/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_col2/Map(decode)_31\n  col2/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_6_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten_3\n  ExternalTransform(beam:transforms:xlang:test:flatten)/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_35\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_36\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_38\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_39\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_40\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_42\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_43\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_44\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_45\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_49\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_50\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_51\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_group_by_key (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f9ece5bb938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_6\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_7_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:gbk)/TestLabel:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at validate_runner_xlang_test.py:103>)_8\n  Map(<lambda at validate_runner_xlang_test.py:103>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_11\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_12\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_14\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_15\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_16\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_18\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_19\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_20\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_21\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_25\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_26\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_27\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... ERROR
test_partition (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f9ece5bb938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:35 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn)_5\n  ExternalTransform(beam:transforms:xlang:test:partition)/Partition(CallableWrapperPartitionFn)/ParDo(ApplyPartitionFnFn)/ParDo(ApplyPartitionFnFn):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>)_6\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:215>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_9_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>)_7\n  ExternalTransform(beam:transforms:xlang:test:partition)/Map(<lambda at expansion_service_test.py:216>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Impulse_20\n  check_even/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/FlatMap(<lambda at core.py:2643>)_21\n  check_even/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Create/Map(decode)_23\n  check_even/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/WindowInto(WindowIntoFn)_24\n  check_even/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/ToVoidKey_25\n  check_even/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_0_27\n  check_even/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/pair_with_1_28\n  check_even/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Flatten_29\n  check_even/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/GroupByKey_30\n  check_even/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Group/Map(_merge_tagged_vals_under_key)_34\n  check_even/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Unkey_35\n  check_even/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_even/Match_36\n  check_even/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Impulse_39\n  check_odd/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/FlatMap(<lambda at core.py:2643>)_40\n  check_odd/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Create/Map(decode)_42\n  check_odd/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/WindowInto(WindowIntoFn)_43\n  check_odd/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/ToVoidKey_44\n  check_odd/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_0_46\n  check_odd/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/pair_with_1_47\n  check_odd/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Flatten_48\n  check_odd/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/GroupByKey_49\n  check_odd/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Group/Map(_merge_tagged_vals_under_key)_53\n  check_odd/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Unkey_54\n  check_odd/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_check_odd/Match_55\n  check_odd/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_prefix (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest) ... WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f9ece5bb938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:21 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n  Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2643>)_4\n  Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n  Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14\n  Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_15\n  Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_16\n  Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'external_10_AppliedPTransform_ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel_3\n  ExternalTransform(beam:transforms:xlang:test:prefix)/TestLabel:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_20\n  assert_that/Create/Impulse:beam:transform:impulse:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2643>)_21\n  assert_that/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_23\n  assert_that/Create/Map(decode):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_24\n  assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_25\n  assert_that/ToVoidKey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_27\n  assert_that/Group/pair_with_0:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_28\n  assert_that/Group/pair_with_1:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_29\n  assert_that/Group/Flatten:beam:transform:flatten:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_30\n  assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_34\n  assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_35\n  assert_that/Unkey:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_36\n  assert_that/Match:beam:transform:pardo:v1\n  must follow: \n  downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok

======================================================================
ERROR: test_multi_input_output_with_sideinput (apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/validate_runner_xlang_test.py",> line 85, in test_multi_input_output_with_sideinput
    TEST_MULTI_URN, None, ValidateRunnerXlangTest.expansion_service)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 554, in __ror__
    result = p.apply(self, pvalueish, label)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/pipeline.py",> line 626, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply
    return m(transform, input, options)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform
    return transform.expand(input)
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/transforms/external.py",> line 330, in expand
    raise RuntimeError(response.error)
RuntimeError: Traceback (most recent call last):
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/runners/portability/expansion_service.py",> line 67, in Expand
    ptransform.PTransform.from_runner_api(request.transform, context))
  File "<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py",> line 685, in from_runner_api
    context)
TypeError: from_runner_api_parameter() takes exactly 2 arguments (3 given)


----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 10 tests in 67.528s

FAILED (errors=1)

> Task :runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython FAILED
> Task :runners:spark:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:job-server:sparkJobServerCleanup

FAILURE: Build completed with 4 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://builds.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 16m 2s
102 actionable tasks: 81 executed, 19 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/kdhcz6ubpwu3s

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org