You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Beam JIRA Bot (Jira)" <ji...@apache.org> on 2020/09/12 17:08:01 UTC
[jira] [Commented] (BEAM-10389) SqlTransform only allows one
registered RowCoder schema
[ https://issues.apache.org/jira/browse/BEAM-10389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17194787#comment-17194787 ]
Beam JIRA Bot commented on BEAM-10389:
--------------------------------------
This issue was marked "stale-P2" and has not received a public comment in 14 days. It is now automatically moved to P3. If you are still affected by it, you can comment and move it back to P2.
> SqlTransform only allows one registered RowCoder schema
> -------------------------------------------------------
>
> Key: BEAM-10389
> URL: https://issues.apache.org/jira/browse/BEAM-10389
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Maximilian Michels
> Priority: P3
>
> The current workflow for using the SqlTransform is:
> {code:python}
> Row = typing.NamedTuple("Row", [("col1", int), ("col2", str)])
> beam.coders.registry.register_coder(Row, beam.coders.RowCoder)
> with self.create_pipeline() as p:
> output = (
> p
> | 'Create' >> beam.Create([Row(x, str(x)) for x in range(5)])
> | 'Sql' >> SqlTransform(
> """SELECT col1, col2 || '*' || col2 as col2,
> power(col1, 2) as col3
> FROM PCOLLECTION))
> {code}
> This works fine, but when multiple row schemas are registered like this:
> {code:python}
> Row = typing.NamedTuple("Row", [("col1", int), ("col2", str)])
> beam.coders.registry.register_coder(Row, beam.coders.RowCoder)
> with self.create_pipeline() as p:
> output = (
> p
> | 'Create' >> beam.Create([Row(x, str(x)) for x in range(5)])
> | 'Sql' >> SqlTransform(
> """SELECT col1, col2 || '*' || col2 as col2,
> power(col1, 2) as col3
> FROM PCOLLECTION))
> output2 = (
> p
> | 'Create2' >> beam.Create([Row2(x, str(x)) for x in range(5)])
> | 'Sql2' >> SqlTransform(
> """SELECT col1, col2 || '*' || col2 as col2,
> power(col1, 2) as col3
> FROM PCOLLECTION
> """))
> {code}
> This yields:
> {noformat}
> RuntimeError: Re-used coder id: ref_Coder_RowCoder_1
> {noformat}
> Source: https://github.com/apache/beam/blob/a8f390704925d3a371b007ccbfcfc28a48b312d1/sdks/python/apache_beam/transforms/external.py#L419
--
This message was sent by Atlassian Jira
(v8.3.4#803005)