You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2019/10/30 15:41:16 UTC
Build failed in Jenkins: beam_LoadTests_Python_Combine_Flink_Batch
#88
See <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/88/display/redirect?page=changes>
Changes:
[ehudm] [BEAM-4132] Support multi-output type inference
[kirillkozlov] Updated PushDown rule and TestTableProvider to support predicate
[kirillkozlov] Updated cost model to favor IOs with filters pushed-down, added more
[ehudm] Add assertions on pcoll types
[ehudm] Don't try to infer recursively
[ehudm] [BEAM-8269] Prefer class decorators over from_callable
[kirillkozlov] Added more tests, bug fixing
[kirillkozlov] Addressed review comments
[kirillkozlov] Fixed style and readability issues
[lcwik] [BEAM-8515, BEAM-2699] Add equals/hashCode to ValueProvider classes.
[lcwik] [BEAM-7230] Cache DataSource instances based upon equivalent
[thw] Fail when pipeline options conversion fails due to incompatible
[github] [BEAM-8517] Fix timezone conversion issue due to usage of different
------------------------------------------
[...truncated 122.73 KB...]
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (3.10.0)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (1.4.1)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (2.8.0)
Requirement already satisfied: pytz>=2018.3 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (2019.3)
Requirement already satisfied: avro<2.0.0,>=1.8.1 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (1.9.1)
Requirement already satisfied: funcsigs<2,>=1.0.2 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (1.0.2)
Requirement already satisfied: futures<4.0.0,>=3.2.0 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (3.3.0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (0.6.8)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (3.6.6)
Requirement already satisfied: pyarrow<0.14.0,>=0.11.1 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (0.13.0)
Requirement already satisfied: cachetools<4,>=3.1.0 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (3.1.1)
Requirement already satisfied: google-apitools<0.5.29,>=0.5.28 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (0.5.28)
Requirement already satisfied: google-cloud-datastore<1.8.0,>=1.7.1 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (1.7.4)
Requirement already satisfied: google-cloud-pubsub<1.1.0,>=0.39.0 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (1.0.2)
Requirement already satisfied: google-cloud-bigquery<1.18.0,>=1.6.0 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (1.17.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (1.0.3)
Requirement already satisfied: google-cloud-bigtable<1.1.0,>=0.31.1 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (1.0.0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore<7.1,>=7.0.1 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (7.0.2)
Requirement already satisfied: nose>=1.3.7 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (1.3.7)
Requirement already satisfied: nose_xunitmp>=0.4.1 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (0.4.1)
Requirement already satisfied: numpy<2,>=1.14.3 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (1.16.5)
Requirement already satisfied: pandas<0.25,>=0.23.4 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (0.24.2)
Requirement already satisfied: parameterized<0.7.0,>=0.6.0 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (0.6.3)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (1.9.0)
Requirement already satisfied: pyyaml<6.0.0,>=3.12 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (5.1.2)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (1.7.0)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (5.1.5)
Requirement already satisfied: six>=1.5.2 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from grpcio<2,>=1.12.1->apache-beam==2.18.0.dev0) (1.12.0)
Requirement already satisfied: enum34>=1.0.4; python_version < "3.4" in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from grpcio<2,>=1.12.1->apache-beam==2.18.0.dev0) (1.1.6)
Requirement already satisfied: docopt in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.18.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.18.0.dev0) (2.22.0)
Requirement already satisfied: pbr>=0.11 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.18.0.dev0) (5.4.3)
Requirement already satisfied: pyasn1>=0.1.7 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from oauth2client<4,>=2.0.1->apache-beam==2.18.0.dev0) (0.4.7)
Requirement already satisfied: pyasn1-modules>=0.0.5 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from oauth2client<4,>=2.0.1->apache-beam==2.18.0.dev0) (0.2.7)
Requirement already satisfied: rsa>=3.1.4 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from oauth2client<4,>=2.0.1->apache-beam==2.18.0.dev0) (4.0)
Requirement already satisfied: setuptools in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from protobuf<4,>=3.5.0.post1->apache-beam==2.18.0.dev0) (41.6.0)
Requirement already satisfied: pyparsing>=2.1.4 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from pydot<2,>=1.2.0->apache-beam==2.18.0.dev0) (2.4.2)
Requirement already satisfied: fasteners>=0.14 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from google-apitools<0.5.29,>=0.5.28->apache-beam==2.18.0.dev0) (0.15)
Requirement already satisfied: google-api-core[grpc]<2.0.0dev,>=1.6.0 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from google-cloud-datastore<1.8.0,>=1.7.1->apache-beam==2.18.0.dev0) (1.14.3)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from google-cloud-pubsub<1.1.0,>=0.39.0->apache-beam==2.18.0.dev0) (0.12.3)
Requirement already satisfied: google-resumable-media<0.5.0dev,>=0.3.1 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from google-cloud-bigquery<1.18.0,>=1.6.0->apache-beam==2.18.0.dev0) (0.4.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.18.0.dev0) (1.6.0)
Requirement already satisfied: monotonic>=0.6; python_version == "2.7" in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from tenacity<6.0,>=5.0.2->apache-beam==2.18.0.dev0) (1.5)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.18.0.dev0) (1.25.6)
Requirement already satisfied: certifi>=2017.4.17 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.18.0.dev0) (2019.9.11)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.18.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.9,>=2.5 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.18.0.dev0) (2.8)
Requirement already satisfied: google-auth<2.0dev,>=0.4.0 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-datastore<1.8.0,>=1.7.1->apache-beam==2.18.0.dev0) (1.6.3)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.18.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam
> Task :sdks:python:apache_beam:testing:load_tests:run
setup.py:191: UserWarning: You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
<https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/local/lib/python2.7/site-packages/setuptools/dist.py>:475: UserWarning: Normalizing '2.18.0.dev' to '2.18.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/sdks/python/apache_beam/__init__.py>:84: UserWarning: You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
testCombineGlobally (apache_beam.testing.load_tests.combine_test.CombineTest) ... ERROR
======================================================================
ERROR: testCombineGlobally (apache_beam.testing.load_tests.combine_test.CombineTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 70, in tearDown
result = self.pipeline.run()
File "<https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 114, in run
state = result.wait_until_finish()
File "<https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 464, in wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline load-tests-python-flink-batch-combine-5-1030150151_2ac580dc-aac2-41b9-b17a-09cdb720890b failed in state FAILED: java.lang.RuntimeException: Error received from SDK harness for instruction 20: Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 173, in _execute
response = task()
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 196, in <lambda>
self._execute(lambda: worker.do_instruction(work), work)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 358, in do_instruction
request.instruction_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 378, in process_bundle
instruction_id, request.process_bundle_descriptor_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in get
self.fns[bundle_descriptor_id],
KeyError: u'1-9'
-------------------- >> begin captured logging << --------------------
root: INFO: Metrics will not be collected
root: INFO: ==================== <function lift_combiners at 0x7f4e4ae67d70> ====================
root: DEBUG: 105 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
root: DEBUG: Stages: ['ref_AppliedPTransform_Read/Impulse_3\n Read/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Split_4\n Read/Split:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/AddRandomKeys_6\n Read/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_8\n Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/GroupByKey_9\n Read/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_13\n Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/RemoveRandomKeys_14\n Read/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/ReadSplits_15\n Read/ReadSplits:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: Start_16\n Measure time: Start:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/KeyWithVoid_18\n Combine with Top 0/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/Precombine\n Combine with Top 0/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/Group\n Combine with Top 0/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/Merge\n Combine with Top 0/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/ExtractOutputs\n Combine with Top 0/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/UnKey_26\n Combine with Top 0/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/DoOnce/Impulse_28\n Combine with Top 0/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/DoOnce/FlatMap(<lambda at core.py:2467>)_29\n Combine with Top 0/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/DoOnce/Map(decode)_31\n Combine with Top 0/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/InjectDefault_32\n Combine with Top 0/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 0_33\n Consume 0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 0_34\n Measure time: End 0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/KeyWithVoid_36\n Combine with Top 1/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/Precombine\n Combine with Top 1/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/Group\n Combine with Top 1/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/Merge\n Combine with Top 1/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/ExtractOutputs\n Combine with Top 1/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/UnKey_44\n Combine with Top 1/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/DoOnce/Impulse_46\n Combine with Top 1/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/DoOnce/FlatMap(<lambda at core.py:2467>)_47\n Combine with Top 1/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/DoOnce/Map(decode)_49\n Combine with Top 1/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/InjectDefault_50\n Combine with Top 1/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 1_51\n Consume 1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 1_52\n Measure time: End 1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/KeyWithVoid_54\n Combine with Top 2/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/Precombine\n Combine with Top 2/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/Group\n Combine with Top 2/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/Merge\n Combine with Top 2/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/ExtractOutputs\n Combine with Top 2/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/UnKey_62\n Combine with Top 2/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/DoOnce/Impulse_64\n Combine with Top 2/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/DoOnce/FlatMap(<lambda at core.py:2467>)_65\n Combine with Top 2/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/DoOnce/Map(decode)_67\n Combine with Top 2/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/InjectDefault_68\n Combine with Top 2/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 2_69\n Consume 2:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 2_70\n Measure time: End 2:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/KeyWithVoid_72\n Combine with Top 3/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/Precombine\n Combine with Top 3/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/Group\n Combine with Top 3/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/Merge\n Combine with Top 3/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/ExtractOutputs\n Combine with Top 3/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/UnKey_80\n Combine with Top 3/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/DoOnce/Impulse_82\n Combine with Top 3/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/DoOnce/FlatMap(<lambda at core.py:2467>)_83\n Combine with Top 3/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/DoOnce/Map(decode)_85\n Combine with Top 3/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/InjectDefault_86\n Combine with Top 3/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 3_87\n Consume 3:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 3_88\n Measure time: End 3:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 4/KeyWithVoid_90\n Combine with Top 4/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 4/CombinePerKey/Precombine\n Combine with Top 4/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 4/CombinePerKey/Group\n Combine with Top 4/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 4/CombinePerKey/Merge\n Combine with Top 4/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 4/CombinePerKey/ExtractOutputs\n Combine with Top 4/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 4/UnKey_98\n Combine with Top 4/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 4/DoOnce/Impulse_100\n Combine with Top 4/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 4/DoOnce/FlatMap(<lambda at core.py:2467>)_101\n Combine with Top 4/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 4/DoOnce/Map(decode)_103\n Combine with Top 4/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 4/InjectDefault_104\n Combine with Top 4/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 4_105\n Consume 4:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 4_106\n Measure time: End 4:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 5/KeyWithVoid_108\n Combine with Top 5/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 5/CombinePerKey/Precombine\n Combine with Top 5/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 5/CombinePerKey/Group\n Combine with Top 5/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 5/CombinePerKey/Merge\n Combine with Top 5/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 5/CombinePerKey/ExtractOutputs\n Combine with Top 5/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 5/UnKey_116\n Combine with Top 5/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 5/DoOnce/Impulse_118\n Combine with Top 5/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 5/DoOnce/FlatMap(<lambda at core.py:2467>)_119\n Combine with Top 5/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 5/DoOnce/Map(decode)_121\n Combine with Top 5/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 5/InjectDefault_122\n Combine with Top 5/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 5_123\n Consume 5:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 5_124\n Measure time: End 5:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 6/KeyWithVoid_126\n Combine with Top 6/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 6/CombinePerKey/Precombine\n Combine with Top 6/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 6/CombinePerKey/Group\n Combine with Top 6/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 6/CombinePerKey/Merge\n Combine with Top 6/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 6/CombinePerKey/ExtractOutputs\n Combine with Top 6/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 6/UnKey_134\n Combine with Top 6/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 6/DoOnce/Impulse_136\n Combine with Top 6/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 6/DoOnce/FlatMap(<lambda at core.py:2467>)_137\n Combine with Top 6/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 6/DoOnce/Map(decode)_139\n Combine with Top 6/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 6/InjectDefault_140\n Combine with Top 6/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 6_141\n Consume 6:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 6_142\n Measure time: End 6:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 7/KeyWithVoid_144\n Combine with Top 7/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 7/CombinePerKey/Precombine\n Combine with Top 7/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 7/CombinePerKey/Group\n Combine with Top 7/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 7/CombinePerKey/Merge\n Combine with Top 7/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 7/CombinePerKey/ExtractOutputs\n Combine with Top 7/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 7/UnKey_152\n Combine with Top 7/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 7/DoOnce/Impulse_154\n Combine with Top 7/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 7/DoOnce/FlatMap(<lambda at core.py:2467>)_155\n Combine with Top 7/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 7/DoOnce/Map(decode)_157\n Combine with Top 7/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 7/InjectDefault_158\n Combine with Top 7/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 7_159\n Consume 7:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 7_160\n Measure time: End 7:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
root: INFO: ==================== <function expand_sdf at 0x7f4e4ae67de8> ====================
root: DEBUG: 105 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
root: DEBUG: Stages: ['ref_AppliedPTransform_Read/Impulse_3\n Read/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Split_4\n Read/Split:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/AddRandomKeys_6\n Read/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_8\n Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/GroupByKey_9\n Read/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_13\n Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/RemoveRandomKeys_14\n Read/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/ReadSplits_15\n Read/ReadSplits:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: Start_16\n Measure time: Start:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/KeyWithVoid_18\n Combine with Top 0/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/Precombine\n Combine with Top 0/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/Group\n Combine with Top 0/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/Merge\n Combine with Top 0/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/ExtractOutputs\n Combine with Top 0/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/UnKey_26\n Combine with Top 0/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/DoOnce/Impulse_28\n Combine with Top 0/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/DoOnce/FlatMap(<lambda at core.py:2467>)_29\n Combine with Top 0/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/DoOnce/Map(decode)_31\n Combine with Top 0/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/InjectDefault_32\n Combine with Top 0/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 0_33\n Consume 0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 0_34\n Measure time: End 0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/KeyWithVoid_36\n Combine with Top 1/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/Precombine\n Combine with Top 1/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/Group\n Combine with Top 1/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/Merge\n Combine with Top 1/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/ExtractOutputs\n Combine with Top 1/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/UnKey_44\n Combine with Top 1/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/DoOnce/Impulse_46\n Combine with Top 1/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/DoOnce/FlatMap(<lambda at core.py:2467>)_47\n Combine with Top 1/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/DoOnce/Map(decode)_49\n Combine with Top 1/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/InjectDefault_50\n Combine with Top 1/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 1_51\n Consume 1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 1_52\n Measure time: End 1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/KeyWithVoid_54\n Combine with Top 2/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/Precombine\n Combine with Top 2/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/Group\n Combine with Top 2/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/Merge\n Combine with Top 2/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/ExtractOutputs\n Combine with Top 2/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/UnKey_62\n Combine with Top 2/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/DoOnce/Impulse_64\n Combine with Top 2/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/DoOnce/FlatMap(<lambda at core.py:2467>)_65\n Combine with Top 2/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/DoOnce/Map(decode)_67\n Combine with Top 2/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/InjectDefault_68\n Combine with Top 2/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 2_69\n Consume 2:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 2_70\n Measure time: End 2:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/KeyWithVoid_72\n Combine with Top 3/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/Precombine\n Combine with Top 3/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/Group\n Combine with Top 3/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/Merge\n Combine with Top 3/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/ExtractOutputs\n Combine with Top 3/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/UnKey_80\n Combine with Top 3/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/DoOnce/Impulse_82\n Combine with Top 3/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/DoOnce/FlatMap(<lambda at core.py:2467>)_83\n Combine with Top 3/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/DoOnce/Map(decode)_85\n Combine with Top 3/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/InjectDefault_86\n Combine with Top 3/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 3_87\n Consume 3:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 3_88\n Measure time: End 3:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 4/KeyWithVoid_90\n Combine with Top 4/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 4/CombinePerKey/Precombine\n Combine with Top 4/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 4/CombinePerKey/Group\n Combine with Top 4/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 4/CombinePerKey/Merge\n Combine with Top 4/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 4/CombinePerKey/ExtractOutputs\n Combine with Top 4/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 4/UnKey_98\n Combine with Top 4/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 4/DoOnce/Impulse_100\n Combine with Top 4/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 4/DoOnce/FlatMap(<lambda at core.py:2467>)_101\n Combine with Top 4/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 4/DoOnce/Map(decode)_103\n Combine with Top 4/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 4/InjectDefault_104\n Combine with Top 4/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 4_105\n Consume 4:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 4_106\n Measure time: End 4:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 5/KeyWithVoid_108\n Combine with Top 5/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 5/CombinePerKey/Precombine\n Combine with Top 5/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 5/CombinePerKey/Group\n Combine with Top 5/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 5/CombinePerKey/Merge\n Combine with Top 5/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 5/CombinePerKey/ExtractOutputs\n Combine with Top 5/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 5/UnKey_116\n Combine with Top 5/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 5/DoOnce/Impulse_118\n Combine with Top 5/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 5/DoOnce/FlatMap(<lambda at core.py:2467>)_119\n Combine with Top 5/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 5/DoOnce/Map(decode)_121\n Combine with Top 5/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 5/InjectDefault_122\n Combine with Top 5/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 5_123\n Consume 5:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 5_124\n Measure time: End 5:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 6/KeyWithVoid_126\n Combine with Top 6/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 6/CombinePerKey/Precombine\n Combine with Top 6/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 6/CombinePerKey/Group\n Combine with Top 6/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 6/CombinePerKey/Merge\n Combine with Top 6/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 6/CombinePerKey/ExtractOutputs\n Combine with Top 6/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 6/UnKey_134\n Combine with Top 6/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 6/DoOnce/Impulse_136\n Combine with Top 6/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 6/DoOnce/FlatMap(<lambda at core.py:2467>)_137\n Combine with Top 6/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 6/DoOnce/Map(decode)_139\n Combine with Top 6/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 6/InjectDefault_140\n Combine with Top 6/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 6_141\n Consume 6:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 6_142\n Measure time: End 6:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 7/KeyWithVoid_144\n Combine with Top 7/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 7/CombinePerKey/Precombine\n Combine with Top 7/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 7/CombinePerKey/Group\n Combine with Top 7/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 7/CombinePerKey/Merge\n Combine with Top 7/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 7/CombinePerKey/ExtractOutputs\n Combine with Top 7/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 7/UnKey_152\n Combine with Top 7/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 7/DoOnce/Impulse_154\n Combine with Top 7/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 7/DoOnce/FlatMap(<lambda at core.py:2467>)_155\n Combine with Top 7/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 7/DoOnce/Map(decode)_157\n Combine with Top 7/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 7/InjectDefault_158\n Combine with Top 7/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 7_159\n Consume 7:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 7_160\n Measure time: End 7:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
root: DEBUG: Runner option 'job_name' was already added
root: DEBUG: Runner option 'runner' was already added
root: DEBUG: Runner option 'temp_location' was already added
root: DEBUG: Runner option 'streaming' was already added
root: DEBUG: Runner option 'environment_cache_millis' was already added
root: DEBUG: Runner option 'files_to_stage' was already added
root: DEBUG: Runner option 'job_endpoint' was already added
root: DEBUG: Runner option 'sdk_worker_parallelism' was already added
root: DEBUG: Runner option 'dataflow_kms_key' was already added
root: DEBUG: Runner option 'enable_streaming_engine' was already added
root: DEBUG: Runner option 'project' was already added
root: DEBUG: Runner option 'zone' was already added
root: DEBUG: Runner option 'experiments' was already added
root: DEBUG: Runner option 'pubsub_root_url' was already added
root: WARNING: Discarding unparseable args: ['--publish_to_big_query=false', '--metrics_dataset=load_test', '--metrics_table=python_flink_batch_combine_5', '--input_options={"num_records": 2500000,"key_size": 10,"value_size": 90}', '--fanout=8', '--top_count=20']
root: INFO: Job state changed to RUNNING
root: DEBUG: org.apache.flink.client.program.ProgramInvocationException: Job failed. (JobID: c805192e4184a09c9404ea562e1f57dc)
at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:262)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:338)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:326)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:301)
at org.apache.flink.client.RemoteExecutor.executePlanWithJars(RemoteExecutor.java:209)
at org.apache.flink.client.RemoteExecutor.executePlan(RemoteExecutor.java:186)
at org.apache.flink.api.java.RemoteEnvironment.execute(RemoteEnvironment.java:173)
at org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator$BatchTranslationContext.execute(FlinkBatchPortablePipelineTranslator.java:191)
at org.apache.beam.runners.flink.FlinkPipelineRunner.runPipelineWithTranslator(FlinkPipelineRunner.java:105)
at org.apache.beam.runners.flink.FlinkPipelineRunner.run(FlinkPipelineRunner.java:81)
at org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:78)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:259)
... 16 more
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction 20: Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 173, in _execute
response = task()
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 196, in <lambda>
self._execute(lambda: worker.do_instruction(work), work)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 358, in do_instruction
request.instruction_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 378, in process_bundle
instruction_id, request.process_bundle_descriptor_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in get
self.fns[bundle_descriptor_id],
KeyError: u'1-9'
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at org.apache.beam.sdk.util.MoreFutures.get(MoreFutures.java:57)
at org.apache.beam.runners.fnexecution.control.SdkHarnessClient$ActiveBundle.close(SdkHarnessClient.java:254)
at org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.$closeResource(FlinkExecutableStageFunction.java:200)
at org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.mapPartition(FlinkExecutableStageFunction.java:200)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:504)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:369)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:530)
... 1 more
Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction 20: Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 173, in _execute
response = task()
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 196, in <lambda>
self._execute(lambda: worker.do_instruction(work), work)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 358, in do_instruction
request.instruction_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 378, in process_bundle
instruction_id, request.process_bundle_descriptor_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in get
self.fns[bundle_descriptor_id],
KeyError: u'1-9'
at org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:157)
at org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:140)
at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onMessage(ServerCalls.java:249)
at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.ForwardingServerCallListener.onMessage(ForwardingServerCallListener.java:33)
at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.Contexts$ContextualizedServerCallListener.onMessage(Contexts.java:76)
at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(ServerCallImpl.java:297)
at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext(ServerImpl.java:738)
at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
root: ERROR: java.lang.RuntimeException: Error received from SDK harness for instruction 20: Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 173, in _execute
response = task()
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 196, in <lambda>
self._execute(lambda: worker.do_instruction(work), work)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 358, in do_instruction
request.instruction_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 378, in process_bundle
instruction_id, request.process_bundle_descriptor_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in get
self.fns[bundle_descriptor_id],
KeyError: u'1-9'
root: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 1 test in 13.956s
FAILED (errors=1)
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 55
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 21s
3 actionable tasks: 2 executed, 1 up-to-date
Publishing build scan...
https://gradle.com/s/vjr5rka3xvdbu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_LoadTests_Python_Combine_Flink_Batch #90
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/90/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Flink_Batch
#89
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/89/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-8398] Upgrade Google Cloud Java Client APIs
[nlofeudo] Avoid polluting Stackdriver logs with noise during autoscaling events.
[kcweaver] fix PubsubJsonClientTest NPE
[nlofeudo] Spotless Java applied
[github] [Go SDK] Delete spammy log in datamgr.go
[kamil.wasilewski] [BEAM-4775] Extract code responsible for parsing MonitoringInfo
[kamil.wasilewski] [BEAM-4775] Expose metrics gathered by FnApiRunner in LocalJobServicer
[kamil.wasilewski] [BEAM-4775] Publish only user metrics in load tests
[kamil.wasilewski] [BEAM-4775] Use result property of MetricResult in load tests
[kirillkozlov] BigQuery should return row fields in the selected order
[bhulette] Bump Python container versions
[robertwb] Cleanup metrics tests.
[boyuanz] Not inject pubsub into Impulse when in fnapi streaming mode.
[tweise] [BEAM-8507] Augment 'flink_master' pipeline option for Python
[github] Remove --flink-master-url alias.
[derek.he] [BEAM-8306] estimate byte size by product count
[github] [BEAM-8474] A microbenchmark for Python FnApiRunner: (#9891)
[robertwb] Remove reference to --flink-master-url.
[robertwb] Revert "Remove --flink-master-url alias."
[mxm] Remove occurrences of flink-master-url and add test for
------------------------------------------
[...truncated 103.74 KB...]
Processing /home/jenkins/.cache/pip/wheels/09/61/a5/7e8f4442b3c3d406ee9eb6c06e1ecbe5625f62f8cb19c08f5b/googledatastore-7.0.2-cp27-none-any.whl
Collecting nose>=1.3.7
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/c4/1f/cd/9250fbf2fcc179e28bb4f7ee26a4fc7525914469d83a4f0c09/nose_xunitmp-0.4.1-cp27-none-any.whl
Collecting numpy<2,>=1.14.3
Using cached https://files.pythonhosted.org/packages/d7/b1/3367ea1f372957f97a6752ec725b87886e12af1415216feec9067e31df70/numpy-1.16.5-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pandas<0.25,>=0.23.4
Using cached https://files.pythonhosted.org/packages/db/83/7d4008ffc2988066ff37f6a0bb6d7b60822367dcb36ba5e39aa7801fda54/pandas-0.24.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting parameterized<0.7.0,>=0.6.0
Using cached https://files.pythonhosted.org/packages/3a/49/75f6dadb09e2f8ace3cdffe0c99a04f1b98dff41fbf9e768665d8b469e29/parameterized-0.6.3-py2.py3-none-any.whl
Collecting pyhamcrest<2.0,>=1.9
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/d9/45/dd/65f0b38450c47cf7e5312883deb97d065e030c5cca0a365030/PyYAML-5.1.2-cp27-cp27mu-linux_x86_64.whl
Collecting requests_mock<2.0,>=1.7
Using cached https://files.pythonhosted.org/packages/8c/f1/66c54a412543b29454102ae74b1454fce2d307b1c36e6bd2e9818394df88/requests_mock-1.7.0-py2.py3-none-any.whl
Collecting tenacity<6.0,>=5.0.2
Using cached https://files.pythonhosted.org/packages/45/67/67bb1db087678bc5c6f20766cf18914dfe37b0b9d4e4c5bb87408460b75f/tenacity-5.1.5-py2.py3-none-any.whl
Requirement already satisfied: six>=1.5.2 in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from grpcio<2,>=1.12.1->apache-beam==2.18.0.dev0) (1.12.0)
Requirement already satisfied: enum34>=1.0.4; python_version < "3.4" in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from grpcio<2,>=1.12.1->apache-beam==2.18.0.dev0) (1.1.6)
Processing /home/jenkins/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e/docopt-0.6.2-py2.py3-none-any.whl
Collecting requests>=2.7.0
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting pbr>=0.11
Using cached https://files.pythonhosted.org/packages/46/a4/d5c83831a3452713e4b4f126149bc4fbda170f7cb16a86a00ce57ce0e9ad/pbr-5.4.3-py2.py3-none-any.whl
Collecting pyasn1>=0.1.7
Using cached https://files.pythonhosted.org/packages/a1/71/8f0d444e3a74e5640a3d5d967c1c6b015da9c655f35b2d308a55d907a517/pyasn1-0.4.7-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Collecting rsa>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools in <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from protobuf<4,>=3.5.0.post1->apache-beam==2.18.0.dev0) (41.6.0)
Collecting pyparsing>=2.1.4
Using cached https://files.pythonhosted.org/packages/11/fa/0160cd525c62d7abd076a070ff02b2b94de589f1a9789774f17d7c54058e/pyparsing-2.4.2-py2.py3-none-any.whl
Collecting fasteners>=0.14
Using cached https://files.pythonhosted.org/packages/18/bd/55eb2d6397b9c0e263af9d091ebdb756b15756029b3cededf6461481bc63/fasteners-0.15-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.6.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/de/3a/83/77a1e18e1a8757186df834b86ce6800120ac9c79cd8ca4091b/grpc_google_iam_v1-0.12.3-cp27-none-any.whl
Collecting google-resumable-media<0.5.0dev,>=0.3.1
Using cached https://files.pythonhosted.org/packages/96/d7/b29a41b01b854480891dfc408211ffb0cc7a2a3d5f15a3b6740ec18c845b/google_resumable_media-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting monotonic>=0.6; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/e0/da/55f51ea951e1b7c63a579c09dd7db825bb730ec1fe9c0180fc77bfb31448/urllib3-1.25.6-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/c5/9b/ed0516cc1f7609fb0217e3057ff4f0f9f3e3ce79a369c6af4a6c5ca25664/google_auth-1.6.3-py2.py3-none-any.whl
Installing collected packages: crcmod, dill, fastavro, future, docopt, urllib3, certifi, chardet, idna, requests, hdfs, httplib2, pbr, funcsigs, mock, pymongo, pyasn1, pyasn1-modules, rsa, oauth2client, pyparsing, pydot, python-dateutil, pytz, avro, pyvcf, typing, numpy, pyarrow, cachetools, monotonic, fasteners, google-apitools, googleapis-common-protos, google-auth, google-api-core, google-cloud-core, google-cloud-datastore, grpc-google-iam-v1, google-cloud-pubsub, google-resumable-media, google-cloud-bigquery, google-cloud-bigtable, proto-google-cloud-datastore-v1, googledatastore, nose, nose-xunitmp, pandas, parameterized, pyhamcrest, pyyaml, requests-mock, tenacity, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.9.1 cachetools-3.1.1 certifi-2019.9.11 chardet-3.0.4 crcmod-1.7 dill-0.3.0 docopt-0.6.2 fastavro-0.21.24 fasteners-0.15 funcsigs-1.0.2 future-0.18.2 google-api-core-1.14.3 google-apitools-0.5.28 google-auth-1.6.3 google-cloud-bigquery-1.17.1 google-cloud-bigtable-1.0.0 google-cloud-core-1.0.3 google-cloud-datastore-1.7.4 google-cloud-pubsub-1.0.2 google-resumable-media-0.4.1 googleapis-common-protos-1.6.0 googledatastore-7.0.2 grpc-google-iam-v1-0.12.3 hdfs-2.5.8 httplib2-0.12.0 idna-2.8 mock-2.0.0 monotonic-1.5 nose-1.3.7 nose-xunitmp-0.4.1 numpy-1.16.5 oauth2client-3.0.0 pandas-0.24.2 parameterized-0.6.3 pbr-5.4.3 proto-google-cloud-datastore-v1-0.90.4 pyarrow-0.13.0 pyasn1-0.4.7 pyasn1-modules-0.2.7 pydot-1.4.1 pyhamcrest-1.9.0 pymongo-3.9.0 pyparsing-2.4.2 python-dateutil-2.8.0 pytz-2019.3 pyvcf-0.6.8 pyyaml-5.1.2 requests-2.22.0 requests-mock-1.7.0 rsa-4.0 tenacity-5.1.5 typing-3.6.6 urllib3-1.25.6
> Task :sdks:python:apache_beam:testing:load_tests:run
setup.py:191: UserWarning: You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
<https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/build/gradleenv/1329484227/local/lib/python2.7/site-packages/setuptools/dist.py>:475: UserWarning: Normalizing '2.18.0.dev' to '2.18.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/sdks/python/apache_beam/__init__.py>:84: UserWarning: You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
testCombineGlobally (apache_beam.testing.load_tests.combine_test.CombineTest) ... ERROR
======================================================================
ERROR: testCombineGlobally (apache_beam.testing.load_tests.combine_test.CombineTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 69, in tearDown
result = self.pipeline.run()
File "<https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 114, in run
state = result.wait_until_finish()
File "<https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 485, in wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline load-tests-python-flink-batch-combine-4-1031150421_ed0950d9-b9ea-4296-8a5b-bf2109f5dc3e failed in state FAILED: java.lang.RuntimeException: Error received from SDK harness for instruction 8: Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 173, in _execute
response = task()
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 196, in <lambda>
self._execute(lambda: worker.do_instruction(work), work)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 358, in do_instruction
request.instruction_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 378, in process_bundle
instruction_id, request.process_bundle_descriptor_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in get
self.fns[bundle_descriptor_id],
KeyError: u'4-2'
-------------------- >> begin captured logging << --------------------
root: INFO: ==================== <function lift_combiners at 0x7f7739466cf8> ====================
root: DEBUG: 57 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
root: DEBUG: Stages: ['ref_AppliedPTransform_Read/Impulse_3\n Read/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Split_4\n Read/Split:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/AddRandomKeys_6\n Read/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_8\n Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/GroupByKey_9\n Read/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_13\n Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/RemoveRandomKeys_14\n Read/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/ReadSplits_15\n Read/ReadSplits:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: Start_16\n Measure time: Start:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/KeyWithVoid_18\n Combine with Top 0/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/Precombine\n Combine with Top 0/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/Group\n Combine with Top 0/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/Merge\n Combine with Top 0/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/ExtractOutputs\n Combine with Top 0/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/UnKey_26\n Combine with Top 0/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/DoOnce/Impulse_28\n Combine with Top 0/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/DoOnce/FlatMap(<lambda at core.py:2467>)_29\n Combine with Top 0/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/DoOnce/Map(decode)_31\n Combine with Top 0/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/InjectDefault_32\n Combine with Top 0/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 0_33\n Consume 0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 0_34\n Measure time: End 0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/KeyWithVoid_36\n Combine with Top 1/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/Precombine\n Combine with Top 1/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/Group\n Combine with Top 1/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/Merge\n Combine with Top 1/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/ExtractOutputs\n Combine with Top 1/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/UnKey_44\n Combine with Top 1/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/DoOnce/Impulse_46\n Combine with Top 1/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/DoOnce/FlatMap(<lambda at core.py:2467>)_47\n Combine with Top 1/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/DoOnce/Map(decode)_49\n Combine with Top 1/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/InjectDefault_50\n Combine with Top 1/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 1_51\n Consume 1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 1_52\n Measure time: End 1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/KeyWithVoid_54\n Combine with Top 2/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/Precombine\n Combine with Top 2/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/Group\n Combine with Top 2/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/Merge\n Combine with Top 2/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/ExtractOutputs\n Combine with Top 2/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/UnKey_62\n Combine with Top 2/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/DoOnce/Impulse_64\n Combine with Top 2/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/DoOnce/FlatMap(<lambda at core.py:2467>)_65\n Combine with Top 2/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/DoOnce/Map(decode)_67\n Combine with Top 2/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/InjectDefault_68\n Combine with Top 2/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 2_69\n Consume 2:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 2_70\n Measure time: End 2:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/KeyWithVoid_72\n Combine with Top 3/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/Precombine\n Combine with Top 3/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/Group\n Combine with Top 3/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/Merge\n Combine with Top 3/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/ExtractOutputs\n Combine with Top 3/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/UnKey_80\n Combine with Top 3/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/DoOnce/Impulse_82\n Combine with Top 3/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/DoOnce/FlatMap(<lambda at core.py:2467>)_83\n Combine with Top 3/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/DoOnce/Map(decode)_85\n Combine with Top 3/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/InjectDefault_86\n Combine with Top 3/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 3_87\n Consume 3:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 3_88\n Measure time: End 3:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
root: INFO: ==================== <function expand_sdf at 0x7f7739466d70> ====================
root: DEBUG: 57 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
root: DEBUG: Stages: ['ref_AppliedPTransform_Read/Impulse_3\n Read/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Split_4\n Read/Split:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/AddRandomKeys_6\n Read/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_8\n Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/GroupByKey_9\n Read/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_13\n Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/Reshuffle/RemoveRandomKeys_14\n Read/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read/ReadSplits_15\n Read/ReadSplits:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: Start_16\n Measure time: Start:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/KeyWithVoid_18\n Combine with Top 0/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/Precombine\n Combine with Top 0/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/Group\n Combine with Top 0/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/Merge\n Combine with Top 0/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 0/CombinePerKey/ExtractOutputs\n Combine with Top 0/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/UnKey_26\n Combine with Top 0/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/DoOnce/Impulse_28\n Combine with Top 0/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/DoOnce/FlatMap(<lambda at core.py:2467>)_29\n Combine with Top 0/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/DoOnce/Map(decode)_31\n Combine with Top 0/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 0/InjectDefault_32\n Combine with Top 0/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 0_33\n Consume 0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 0_34\n Measure time: End 0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/KeyWithVoid_36\n Combine with Top 1/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/Precombine\n Combine with Top 1/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/Group\n Combine with Top 1/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/Merge\n Combine with Top 1/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 1/CombinePerKey/ExtractOutputs\n Combine with Top 1/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/UnKey_44\n Combine with Top 1/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/DoOnce/Impulse_46\n Combine with Top 1/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/DoOnce/FlatMap(<lambda at core.py:2467>)_47\n Combine with Top 1/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/DoOnce/Map(decode)_49\n Combine with Top 1/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 1/InjectDefault_50\n Combine with Top 1/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 1_51\n Consume 1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 1_52\n Measure time: End 1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/KeyWithVoid_54\n Combine with Top 2/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/Precombine\n Combine with Top 2/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/Group\n Combine with Top 2/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/Merge\n Combine with Top 2/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 2/CombinePerKey/ExtractOutputs\n Combine with Top 2/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/UnKey_62\n Combine with Top 2/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/DoOnce/Impulse_64\n Combine with Top 2/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/DoOnce/FlatMap(<lambda at core.py:2467>)_65\n Combine with Top 2/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/DoOnce/Map(decode)_67\n Combine with Top 2/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 2/InjectDefault_68\n Combine with Top 2/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 2_69\n Consume 2:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 2_70\n Measure time: End 2:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/KeyWithVoid_72\n Combine with Top 3/KeyWithVoid:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/Precombine\n Combine with Top 3/CombinePerKey/Precombine:beam:transform:combine_per_key_precombine:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/Group\n Combine with Top 3/CombinePerKey/Group:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/Merge\n Combine with Top 3/CombinePerKey/Merge:beam:transform:combine_per_key_merge_accumulators:v1\n must follow: \n downstream_side_inputs: <unknown>', 'Combine with Top 3/CombinePerKey/ExtractOutputs\n Combine with Top 3/CombinePerKey/ExtractOutputs:beam:transform:combine_per_key_extract_outputs:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/UnKey_80\n Combine with Top 3/UnKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/DoOnce/Impulse_82\n Combine with Top 3/DoOnce/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/DoOnce/FlatMap(<lambda at core.py:2467>)_83\n Combine with Top 3/DoOnce/FlatMap(<lambda at core.py:2467>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/DoOnce/Map(decode)_85\n Combine with Top 3/DoOnce/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Combine with Top 3/InjectDefault_86\n Combine with Top 3/InjectDefault:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume 3_87\n Consume 3:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End 3_88\n Measure time: End 3:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
root: DEBUG: Runner option 'job_name' was already added
root: DEBUG: Runner option 'runner' was already added
root: DEBUG: Runner option 'temp_location' was already added
root: DEBUG: Runner option 'streaming' was already added
root: DEBUG: Runner option 'environment_cache_millis' was already added
root: DEBUG: Runner option 'files_to_stage' was already added
root: DEBUG: Runner option 'job_endpoint' was already added
root: DEBUG: Runner option 'sdk_worker_parallelism' was already added
root: DEBUG: Runner option 'dataflow_kms_key' was already added
root: DEBUG: Runner option 'enable_streaming_engine' was already added
root: DEBUG: Runner option 'project' was already added
root: DEBUG: Runner option 'zone' was already added
root: DEBUG: Runner option 'experiments' was already added
root: DEBUG: Runner option 'pubsub_root_url' was already added
root: WARNING: Discarding unparseable args: ['--publish_to_big_query=false', '--metrics_dataset=load_test', '--metrics_table=python_flink_batch_combine_4', '--input_options={"num_records": 5000000,"key_size": 10,"value_size": 90}', '--fanout=4', '--top_count=20']
root: INFO: Job state changed to RUNNING
root: DEBUG: org.apache.flink.client.program.ProgramInvocationException: Job failed. (JobID: 59056cc0a688346cfa74e58d19fef6f7)
at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:262)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:338)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:326)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:301)
at org.apache.flink.client.RemoteExecutor.executePlanWithJars(RemoteExecutor.java:209)
at org.apache.flink.client.RemoteExecutor.executePlan(RemoteExecutor.java:186)
at org.apache.flink.api.java.RemoteEnvironment.execute(RemoteEnvironment.java:173)
at org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator$BatchTranslationContext.execute(FlinkBatchPortablePipelineTranslator.java:191)
at org.apache.beam.runners.flink.FlinkPipelineRunner.runPipelineWithTranslator(FlinkPipelineRunner.java:105)
at org.apache.beam.runners.flink.FlinkPipelineRunner.run(FlinkPipelineRunner.java:81)
at org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:78)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:259)
... 16 more
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction 8: Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 173, in _execute
response = task()
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 196, in <lambda>
self._execute(lambda: worker.do_instruction(work), work)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 358, in do_instruction
request.instruction_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 378, in process_bundle
instruction_id, request.process_bundle_descriptor_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in get
self.fns[bundle_descriptor_id],
KeyError: u'4-2'
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at org.apache.beam.sdk.util.MoreFutures.get(MoreFutures.java:57)
at org.apache.beam.runners.fnexecution.control.SdkHarnessClient$ActiveBundle.close(SdkHarnessClient.java:254)
at org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.$closeResource(FlinkExecutableStageFunction.java:200)
at org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.mapPartition(FlinkExecutableStageFunction.java:200)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:504)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:369)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:530)
... 1 more
Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction 8: Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 173, in _execute
response = task()
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 196, in <lambda>
self._execute(lambda: worker.do_instruction(work), work)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 358, in do_instruction
request.instruction_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 378, in process_bundle
instruction_id, request.process_bundle_descriptor_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in get
self.fns[bundle_descriptor_id],
KeyError: u'4-2'
at org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:157)
at org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:140)
at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onMessage(ServerCalls.java:249)
at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.ForwardingServerCallListener.onMessage(ForwardingServerCallListener.java:33)
at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.Contexts$ContextualizedServerCallListener.onMessage(Contexts.java:76)
at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(ServerCallImpl.java:297)
at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext(ServerImpl.java:738)
at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
root: ERROR: java.lang.RuntimeException: Error received from SDK harness for instruction 8: Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 173, in _execute
response = task()
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 196, in <lambda>
self._execute(lambda: worker.do_instruction(work), work)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 358, in do_instruction
request.instruction_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 378, in process_bundle
instruction_id, request.process_bundle_descriptor_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in get
self.fns[bundle_descriptor_id],
KeyError: u'4-2'
root: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 1 test in 56.372s
FAILED (errors=1)
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_LoadTests_Python_Combine_Flink_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 55
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 23s
3 actionable tasks: 3 executed
Publishing build scan...
https://gradle.com/s/q7c7eaguds6ea
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org