[GitHub] [beam] damccorm opened a new issue, #20526: InteractiveRunner cannot execute pipeline with cross-language transform

[GitBox <gi...@apache.org>]

damccorm opened a new issue, #20526:
URL: https://github.com/apache/beam/issues/20526

   The InteractiveRunner crashes when given a pipeline that includes a cross-language transform.
   
   Here's the example I tried to run in a jupyter notebook:
   ```
   
   p = beam.Pipeline(InteractiveRunner())
   pc = (p | SqlTransform("""SELECT
               CAST(1 AS INT)
   AS `id`,
               CAST('foo' AS VARCHAR) AS `str`,
               CAST(3.14  AS DOUBLE) AS `flt`"""))
   df
   = interactive_beam.collect(pc)
   
   ```
   
   
   The problem occurs when [pipeline_fragment.py](https://github.com/apache/beam/blob/dce1eb83b8d5137c56ac58568820c24bd8fda526/sdks/python/apache_beam/runners/interactive/pipeline_fragment.py#L66) creates a copy of the pipeline by [writing it to proto and reading it back](https://github.com/apache/beam/blob/dce1eb83b8d5137c56ac58568820c24bd8fda526/sdks/python/apache_beam/runners/interactive/pipeline_fragment.py#L120). Reading it back fails because some of the pipeline is not written in Python.
   
   Imported from Jira [BEAM-10708](https://issues.apache.org/jira/browse/BEAM-10708). Original Jira may contain additional context.
   Reported by: bhulette.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org