You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Udi Meiri <eh...@google.com> on 2019/04/16 17:57:10 UTC

pickler.py issue with nested classes

I was looking at migrating unit tests to pytest and found this test which
doesn't pass:
https://gist.github.com/udim/a71fcb278b56a9a5b7962f4588e14efb (stack
overflow)
(requires installing python3.7 and "python3.7 -m pip install pytest".)
The same command passes with python2.7 and python3.5.

I tried isolating the issue and created a test case which fails similarly
on all Python versions I tried (2.7, 3.5, 3.7):

def test_local_nested_class(self):
  class LocalNestedClass(object):
    def __init__(self, data):
      # TODO: commenting out the call to __init__ makes the test pass
      super(LocalNestedClass, self).__init__()
      self.data = data

  self.assertEqual('abc', loads(dumps(LocalNestedClass('abc'))).data)

(added to PicklerTest)

Any ideas why this fails, and why removing the call to
super(...).__init__() makes a difference?
Is DataflowRunnerTest::test_remote_runner_display_data trying to do
something that's not supposed to work?

Re: pickler.py issue with nested classes

Posted by Udi Meiri <eh...@google.com>.
Not sure: my case is using a nested class and the error is a stack overflow
(or infinite recursion detection is triggered).

It is odd though that they have the same workaround.

Re: pickler.py issue with nested classes

Posted by Valentyn Tymofieiev <va...@google.com>.
This looks very similar to https://github.com/uqfoundation/dill/issues/300,
however we observed that bug on Python 3, and not on Python 2.7.

On Tue, Apr 16, 2019 at 10:58 AM Udi Meiri <eh...@google.com> wrote:

> I was looking at migrating unit tests to pytest and found this test which
> doesn't pass:
> https://gist.github.com/udim/a71fcb278b56a9a5b7962f4588e14efb (stack
> overflow)
> (requires installing python3.7 and "python3.7 -m pip install pytest".)
> The same command passes with python2.7 and python3.5.
>
> I tried isolating the issue and created a test case which fails similarly
> on all Python versions I tried (2.7, 3.5, 3.7):
>
> def test_local_nested_class(self):
>   class LocalNestedClass(object):
>     def __init__(self, data):
>       # TODO: commenting out the call to __init__ makes the test pass
>       super(LocalNestedClass, self).__init__()
>       self.data = data
>
>   self.assertEqual('abc', loads(dumps(LocalNestedClass('abc'))).data)
>
> (added to PicklerTest)
>
> Any ideas why this fails, and why removing the call to
> super(...).__init__() makes a difference?
> Is DataflowRunnerTest::test_remote_runner_display_data trying to do
> something that's not supposed to work?
>