Re: Python question about save_main_session

[Chad Dombrova <ch...@gmail.com>]

Another solution may be writing a custom dill reducer for the object that
is triggering this recursion.

-chad


On Fri, Aug 23, 2019 at 6:52 PM Udi Meiri <eh...@google.com> wrote:

> Yeah that sounds like the sanest way forward.
> Lots of these tests are running standalone examples so they might need SMS
> if you run them directly.
> (example streaming_wordcount_it_test.py running streaming_wordcount.py)
>
> On Fri, Aug 23, 2019 at 3:17 PM Robert Bradshaw <ro...@google.com>
> wrote:
>
>> I suggest re-writing the test to avoid save_main_session.
>>
>> On Fri, Aug 23, 2019 at 11:57 AM Udi Meiri <eh...@google.com> wrote:
>>
>>> Hi,
>>> I'm trying to get pytest with the xdist plugin to run Beam tests. The
>>> issue is with save_main_session and a dependency of pytest-xdist called
>>> execnet, which triggers this error:
>>>
>>> *apache_beam/examples/complete/tfidf.py*:212: in run*    output | 'write' >> WriteToText(known_args.output)**apache_beam/pipeline.py*:426: in __exit__*    self.run().wait_until_finish()**apache_beam/pipeline.py*:406: in run*    self._options).run(False)**apache_beam/pipeline.py*:416: in run*    pickler.dump_session(os.path.join(tmpdir, 'main_session.pickle'))**apache_beam/internal/pickler.py*:282: in dump_session*    dill.load_session(file_path)**../../../../virtualenvs/beam-py35/lib/python3.5/site-packages/dill/_dill.py*:410: in load_session*    module = unpickler.load()**../../../../virtualenvs/beam-py35/lib/python3.5/site-packages/execnet/gateway_base.py*:130: in __getattr__*    locs = self._importdef.get(name)**../../../../virtualenvs/beam-py35/lib/python3.5/site-packages/execnet/gateway_base.py*:130: in __getattr__*    locs = self._importdef.get(name)**../../../../virtualenvs/beam-py35/lib/python3.5/site-packages/execnet/gateway_base.py*:130: in __getattr__*    locs = self._importdef.get(name)**E   RecursionError: maximum recursion depth exceeded*
>>> !!! Recursion detected (same locals & position)
>>>
>>>
>>> Does anyone on this list have experience with these kinds of errors? Any
>>> workarounds I can use? (can we handle this module specially / can we
>>> exclude it from main session?)
>>>
>>