[jira] [Updated] (BEAM-10020) Java transform executed as Python transform?

["Pawel Urbanowicz (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pawel Urbanowicz updated BEAM-10020:
------------------------------------
    Affects Version/s:     (was: 2.21.0)

> Java transform executed as Python transform?
> --------------------------------------------
>
>                 Key: BEAM-10020
>                 URL: https://issues.apache.org/jira/browse/BEAM-10020
>             Project: Beam
>          Issue Type: Bug
>          Components: jobserver, runner-flink
>            Reporter: Pawel Urbanowicz
>            Priority: P2
>
> Hey, I created a transform method in Java and now I want to use it in Python using Cross-language. I got pretty stuck with the following problem:
> *p*
> *| GenerateSequence(...)* 
> *|ExternalTransform(...) => is working like a charm* 
>  
>  *p* 
> *| Create(...)*
> *| ExternalTransform(...) => getting assert pardo_payload.do_fn.urn == python_urns.PICKLED_DOFN_INFO* 
> Based on [https://www.mail-archive.com/user@beam.apache.org/msg04887.html]
> it seems like a Create transform is being registered as a Java transform but executed as Python transform. 
>  
> Traceback (most recent call last): File "/Users/urban/projects/beam/sdks/python/apache_beam/io/external/snowflake_test.py", line 92, in test_snowflake_write_read self.run_write() File "/Users/urban/projects/beam/sdks/python/apache_beam/io/external/snowflake_test.py", line 129, in run_write expansion_service=self.expansion_service, File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line 528, in __exit__ self.run().wait_until_finish() File "/Users/urban/projects/beam/sdks/python/apache_beam/testing/test_pipeline.py", line 112, in run False if self.not_use_test_runner_api else test_runner_api)) File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line 501, in run self._options).run(False) File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line 858, in from_runner_api p.transforms_stack = [context.transforms.get_by_id(root_transform_id)] File "/Users/urban/projects/beam/sdks/python/apache_beam/runners/pipeline_context.py", line 103, in get_by_id self._id_to_proto[id], self._pipeline_context) File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line 1231, in from_runner_api part = context.transforms.get_by_id(transform_id) File "/Users/urban/projects/beam/sdks/python/apache_beam/runners/pipeline_context.py", line 103, in get_by_id self._id_to_proto[id], self._pipeline_context) File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line 1231, in from_runner_api part = context.transforms.get_by_id(transform_id) File "/Users/urban/projects/beam/sdks/python/apache_beam/runners/pipeline_context.py", line 103, in get_by_id self._id_to_proto[id], self._pipeline_context) File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line 1231, in from_runner_api part = context.transforms.get_by_id(transform_id) File "/Users/urban/projects/beam/sdks/python/apache_beam/runners/pipeline_context.py", line 103, in get_by_id self._id_to_proto[id], self._pipeline_context) File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line 1231, in from_runner_api part = context.transforms.get_by_id(transform_id) File "/Users/urban/projects/beam/sdks/python/apache_beam/runners/pipeline_context.py", line 103, in get_by_id self._id_to_proto[id], self._pipeline_context) File "/Users/urban/projects/beam/sdks/python/apache_beam/pipeline.py", line 1170, in from_runner_api transform = ptransform.PTransform.from_runner_api(proto, context) File "/Users/urban/projects/beam/sdks/python/apache_beam/transforms/ptransform.py", line 685, in from_runner_api context) File "/Users/urban/projects/beam/sdks/python/apache_beam/transforms/core.py", line 1380, in from_runner_api_parameter assert pardo_payload.do_fn.urn == python_urns.PICKLED_DOFN_INFO AssertionError
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)