[jira] [Comment Edited] (BEAM-5617) Side inputs don't work on Python 3

["Valentyn Tymofieiev (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16657654#comment-16657654 ] 

Valentyn Tymofieiev edited comment on BEAM-5617 at 10/25/18 8:48 PM:
---------------------------------------------------------------------

I've been examining the execution path for test_iterable_side_input on Python 3 and comparing it Python 2 by looking at call stack. With the help of 'sys.setprofile()' I intercepted each function call, and printed a function/method name, arguments, return value and the caller, see [1].

The first location I noticed where execution starts to diverge is in pickler.py[2], sending https://github.com/apache/beam/pull/6764. The test still does not pass though.

[1] https://github.com/apache/beam/compare/master...tvalentyn:py3_sideinput_investigations?expand=1
[2] https://github.com/apache/beam/blob/bd8ff7c63768c266ef03b9ad026ac1649e958ed2/sdks/python/apache_beam/internal/pickler.py#L58



was (Author: tvalentyn):
I've been examining the execution path for test_iterable_side_input on Python 3 and comparing it Python 2 by looking at call stack. With the help of 'sys.setprofile()' I intercepted each function call, and printed a function/method name, arguments, return value and the caller.

The first location I noticed where execution starts to diverge is in pickler.py[1], sending https://github.com/apache/beam/pull/6764. The test still does not pass though.

https://github.com/apache/beam/blob/bd8ff7c63768c266ef03b9ad026ac1649e958ed2/sdks/python/apache_beam/internal/pickler.py#L58

> Side inputs don't work on Python 3 
> -----------------------------------
>
>                 Key: BEAM-5617
>                 URL: https://issues.apache.org/jira/browse/BEAM-5617
>             Project: Beam
>          Issue Type: Sub-task
>          Components: sdk-py-core
>            Reporter: Valentyn Tymofieiev
>            Assignee: Valentyn Tymofieiev
>            Priority: Major
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> ======================================================================
> ERROR: test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/runners/common.py", line 677, in process
>     self.do_fn_invoker.invoke_process(windowed_value)
>   File "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/runners/common.py", line 414, in invoke_process
>     windowed_value, self.process_method(windowed_value.value))
>   File "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/transforms/core.py", line 1068, in <lambda>
>     wrapper = lambda x: [fn(x)]
>   File "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/testing/util.py", line 119, in _equal
>     'Failed assert: %r == %r' % (sorted_expected, sorted_actual))
> apache_beam.testing.util.BeamAssertException: Failed assert: [3, 4, 6, 8] == []



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)