You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Thomas Weise (JIRA)" <ji...@apache.org> on 2019/04/21 16:47:00 UTC
[jira] [Commented] (BEAM-7127) Timer parameter not supported in
timer callback
[ https://issues.apache.org/jira/browse/BEAM-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822730#comment-16822730 ]
Thomas Weise commented on BEAM-7127:
------------------------------------
{code:java}
File "/Users/tweise/src/beam/sdks/python/apache_beam/examples/flink/flink_state.py", line 78, in run
| 'statefulCount' >> beam.ParDo(StateTimerFn())
File "apache_beam/transforms/core.py", line 979, in __init__
super(ParDo, self).__init__(fn, *args, **kwargs)
File "apache_beam/transforms/ptransform.py", line 689, in __init__
self.fn = pickler.loads(pickler.dumps(self.fn))
File "apache_beam/internal/pickler.py", line 230, in dumps
s = dill.dumps(o)
File "/Users/tweise/python-ve/beam/lib/python2.7/site-packages/dill/_dill.py", line 294, in dumps
dump(obj, file, protocol, byref, fmode, recurse)#, strictio)
File "/Users/tweise/python-ve/beam/lib/python2.7/site-packages/dill/_dill.py", line 287, in dump
pik.dump(obj)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 396, in save_reduce
save(cls)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "apache_beam/internal/pickler.py", line 107, in wrapper
return fun(pickler, obj)
File "/Users/tweise/python-ve/beam/lib/python2.7/site-packages/dill/_dill.py", line 1315, in save_type
obj.__bases__, _dict), obj=obj)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 401, in save_reduce
save(args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 562, in save_tuple
save(element)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "apache_beam/internal/pickler.py", line 198, in new_save_module_dict
return old_save_module_dict(pickler, obj)
File "/Users/tweise/python-ve/beam/lib/python2.7/site-packages/dill/_dill.py", line 902, in save_module_dict
StockPickler.save_dict(pickler, obj)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 681, in _batch_setitems
save(v)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/Users/tweise/python-ve/beam/lib/python2.7/site-packages/dill/_dill.py", line 1394, in save_function
obj.__dict__), obj=obj)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 405, in save_reduce
self.memoize(obj)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 244, in memoize
assert id(obj) not in self.memo
AssertionError{code}
> Timer parameter not supported in timer callback
> -----------------------------------------------
>
> Key: BEAM-7127
> URL: https://issues.apache.org/jira/browse/BEAM-7127
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-harness
> Affects Versions: 2.12.0
> Reporter: Thomas Weise
> Priority: Major
> Labels: portability
>
> Referencing the timer in its on_timer callback produces a recursive pickle error.
> {code:java}
> @userstate.on_timer(timer_spec)
> def process_timer(self, timer_1=beam.DoFn.TimerParam(timer_spec)):
> {code}
> Unit test: [https://github.com/apache/beam/blob/cbe4dfbdbe5d0da5152568853ee5e17334dd1b54/sdks/python/apache_beam/transforms/userstate_test.py#L67]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)