You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/04 03:55:46 UTC
[GitHub] [beam] kennknowles opened a new issue, #19540: DirectRunner GameStats fails, ValueError: PCollection of size 2 with more than one element accessed as a singleton view. First two elements encountered are "181.611538462", "9.1". [while running 'CalculateSpammyUsers/ProcessAndFilter']
kennknowles opened a new issue, #19540:
URL: https://github.com/apache/beam/issues/19540
GameStats example is failing with following logs
```
*****************************************************
* Running GameStats with DirectRunner
*****************************************************
INFO:root:Missing
pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py:1137:
BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options
will not be supported
method_to_use = self._compute_method(p, p.options)
INFO:root:Running pipeline
with DirectRunner.
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.client:Refreshing
access_token
INFO:root:Created table google.com:clouddfe.goenka_python_validations.game_stats_teams
with schema <TableSchema
fields: [<TableFieldSchema
fields: []
mode: u'NULLABLE'
name: u'processing_time'
type:
u'STRING'>, <TableFieldSchema
fields: []
mode: u'NULLABLE'
name: u'total_score'
type: u'INTEGER'>,
<TableFieldSchema
fields: []
mode: u'NULLABLE'
name: u'window_start'
type: u'STRING'>, <TableFieldSchema
fields:
[]
mode: u'NULLABLE'
name: u'team'
type: u'STRING'>]>. Result: <Table
creationTime: 1559341252301
etag:
u'tEQeN0Sr4FvttBRZuXn7zw=='
id: u'google.com:clouddfe:goenka_python_validations.game_stats_teams'
kind:
u'bigquery#table'
lastModifiedTime: 1559341252336
location: u'US'
numBytes: 0
numLongTermBytes:
0
numRows: 0
schema: <TableSchema
fields: [<TableFieldSchema
fields: []
mode: u'NULLABLE'
name:
u'processing_time'
type: u'STRING'>, <TableFieldSchema
fields: []
mode: u'NULLABLE'
name: u'total_score'
type:
u'INTEGER'>, <TableFieldSchema
fields: []
mode: u'NULLABLE'
name: u'window_start'
type: u'STRING'>,
<TableFieldSchema
fields: []
mode: u'NULLABLE'
name: u'team'
type: u'STRING'>]>
selfLink: u'https://www.googleapis.com/bigquery/v2/projects/google.com:clouddfe/datasets/goenka_python_validations/tables/game_stats_teams'
tableReference:
<TableReference
datasetId: u'goenka_python_validations'
projectId: u'google.com:clouddfe'
tableId:
u'game_stats_teams'>
type: u'TABLE'>.
INFO:oauth2client.transport:Attempting refresh to obtain initial
access_token
INFO:oauth2client.client:Refreshing access_token
ERROR:root:Exception at bundle <apache_beam.runners.direct.bundle_factory._Bundle
object at 0x7fb0f1cfb6d0>, due to an exception.
Traceback (most recent call last):
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 343, in call
finish_state)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 380, in attempt_call
evaluator.process_element(value)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py",
line 633, in process_element
self.runner.process(element)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 753, in process
self._reraise_augmented(exn)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 807, in _reraise_augmented
raise_with_traceback(new_exn)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 751, in process
return self.do_fn_invoker.invoke_process(windowed_value)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 564, in invoke_process
windowed_value, additional_args, additional_kwargs, output_processor)
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 571, in _invoke_process_per_window
side_inputs = [si[window] for si in self.side_inputs]
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
line 65, in __getitem__
_FilteringIterable(self._iterable, target_window), self._view_options)
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 346, in _from_runtime_iterable
return options['data'].view_fn(it)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 323, in <lambda>
lambda iterable: from_runtime_iterable(iterable, view_options))
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 433, in _from_runtime_iterable
len(head), str(head[0]), str(head[1])))
ValueError: PCollection
of size 2 with more than one element accessed as a singleton view. First two elements encountered are
"181.611538462", "9.1". [while running 'CalculateSpammyUsers/ProcessAndFilter']
ERROR:root:Exception
at bundle <apache_beam.runners.direct.bundle_factory._Bundle object at 0x7fb0f1cfb6d0>, due to an exception.
Traceback
(most recent call last):
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 343, in call
finish_state)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 380, in attempt_call
evaluator.process_element(value)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py",
line 633, in process_element
self.runner.process(element)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 753, in process
self._reraise_augmented(exn)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 807, in _reraise_augmented
raise_with_traceback(new_exn)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 751, in process
return self.do_fn_invoker.invoke_process(windowed_value)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 564, in invoke_process
windowed_value, additional_args, additional_kwargs, output_processor)
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 571, in _invoke_process_per_window
side_inputs = [si[window] for si in self.side_inputs]
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
line 65, in __getitem__
_FilteringIterable(self._iterable, target_window), self._view_options)
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 346, in _from_runtime_iterable
return options['data'].view_fn(it)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 323, in <lambda>
lambda iterable: from_runtime_iterable(iterable, view_options))
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 433, in _from_runtime_iterable
len(head), str(head[0]), str(head[1])))
ValueError: PCollection
of size 2 with more than one element accessed as a singleton view. First two elements encountered are
"181.611538462", "9.1". [while running 'CalculateSpammyUsers/ProcessAndFilter']
ERROR:root:Exception
at bundle <apache_beam.runners.direct.bundle_factory._Bundle object at 0x7fb0f1cfb6d0>, due to an exception.
Traceback
(most recent call last):
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 343, in call
finish_state)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 380, in attempt_call
evaluator.process_element(value)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py",
line 633, in process_element
self.runner.process(element)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 753, in process
self._reraise_augmented(exn)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 807, in _reraise_augmented
raise_with_traceback(new_exn)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 751, in process
return self.do_fn_invoker.invoke_process(windowed_value)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 564, in invoke_process
windowed_value, additional_args, additional_kwargs, output_processor)
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 571, in _invoke_process_per_window
side_inputs = [si[window] for si in self.side_inputs]
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
line 65, in __getitem__
_FilteringIterable(self._iterable, target_window), self._view_options)
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 346, in _from_runtime_iterable
return options['data'].view_fn(it)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 323, in <lambda>
lambda iterable: from_runtime_iterable(iterable, view_options))
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 433, in _from_runtime_iterable
len(head), str(head[0]), str(head[1])))
ValueError: PCollection
of size 2 with more than one element accessed as a singleton view. First two elements encountered are
"181.611538462", "9.1". [while running 'CalculateSpammyUsers/ProcessAndFilter']
ERROR:root:Exception
at bundle <apache_beam.runners.direct.bundle_factory._Bundle object at 0x7fb0f1cfb6d0>, due to an exception.
Traceback
(most recent call last):
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 343, in call
finish_state)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 380, in attempt_call
evaluator.process_element(value)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py",
line 633, in process_element
self.runner.process(element)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 753, in process
self._reraise_augmented(exn)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 807, in _reraise_augmented
raise_with_traceback(new_exn)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 751, in process
return self.do_fn_invoker.invoke_process(windowed_value)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 564, in invoke_process
windowed_value, additional_args, additional_kwargs, output_processor)
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 571, in _invoke_process_per_window
side_inputs = [si[window] for si in self.side_inputs]
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
line 65, in __getitem__
_FilteringIterable(self._iterable, target_window), self._view_options)
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 346, in _from_runtime_iterable
return options['data'].view_fn(it)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 323, in <lambda>
lambda iterable: from_runtime_iterable(iterable, view_options))
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 433, in _from_runtime_iterable
len(head), str(head[0]), str(head[1])))
ValueError: PCollection
of size 2 with more than one element accessed as a singleton view. First two elements encountered are
"181.611538462", "9.1". [while running 'CalculateSpammyUsers/ProcessAndFilter']
ERROR:root:Giving
up after 4 attempts.
WARNING:root:A task failed with exception: PCollection of size 2 with more than
one element accessed as a singleton view. First two elements encountered are "181.611538462", "9.1".
[while running 'CalculateSpammyUsers/ProcessAndFilter']
INFO:oauth2client.transport:Attempting refresh
to obtain initial access_token
INFO:oauth2client.client:Refreshing access_token
Traceback (most recent
call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"__main__", fname,
loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/lib/python2.7/site-packages/apache_beam/examples/complete/game/game_stats.py",
line 399, in <module>
run()
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/lib/python2.7/site-packages/apache_beam/examples/complete/game/game_stats.py",
line 393, in run
}, options.view_as(GoogleCloudOptions).project))
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pipeline.py",
line 426, in __exit__
self.run().wait_until_finish()
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/direct_runner.py",
line 430, in wait_until_finish
self._executor.await_completion()
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 400, in await_completion
self._executor.await_completion()
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 446, in await_completion
raise_(t, v, tb)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 343, in call
finish_state)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py",
line 380, in attempt_call
evaluator.process_element(value)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py",
line 633, in process_element
self.runner.process(element)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 753, in process
self._reraise_augmented(exn)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 807, in _reraise_augmented
raise_with_traceback(new_exn)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 751, in process
return self.do_fn_invoker.invoke_process(windowed_value)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 564, in invoke_process
windowed_value, additional_args, additional_kwargs, output_processor)
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/runners/common.py",
line 571, in _invoke_process_per_window
side_inputs = [si[window] for si in self.side_inputs]
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/transforms/sideinputs.py",
line 65, in __getitem__
_FilteringIterable(self._iterable, target_window), self._view_options)
File
"/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 346, in _from_runtime_iterable
return options['data'].view_fn(it)
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 323, in <lambda>
lambda iterable: from_runtime_iterable(iterable, view_options))
File "/usr/local/google/home/goenka/rc_validations/beam_env_python2.7/local/lib/python2.7/site-packages/apache_beam/pvalue.py",
line 433, in _from_runtime_iterable
len(head), str(head[0]), str(head[1])))
ValueError: PCollection
of size 2 with more than one element accessed as a singleton view. First two elements encountered are
"181.611538462", "9.1". [while running 'CalculateSpammyUsers/ProcessAndFilter']
```
Imported from Jira [BEAM-7471](https://issues.apache.org/jira/browse/BEAM-7471). Original Jira may contain additional context.
Reported by: angoenka.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org