[jira] [Commented] (BEAM-3595) Normalize URNs across SDKs and runners.

["Chad Dombrova (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-3595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16883566#comment-16883566 ] 

Chad Dombrova commented on BEAM-3595:
-------------------------------------

I see that this is marked as fixed, but when using v2.13.0 of the python sdk I get bad a bad urn for pardo: "urn:beam:transform:pardo:v1" (also, there's a least a dozen mentions of this ticket in the source code).  In order to read my pipeline back into Java I have to remove the urn: prefix.  The urns for other transforms don't include the urn: prefix. 

Here's a bad pardo message:

{noformat}
  transforms {
    key: "ref_AppliedPTransform_Sleep_4"
    value {
      spec {
        urn: "urn:beam:transform:pardo:v1"
        payload: "\n\204\002\n\201\002\n beam:dofn:pickled_python_info:v1\032\3Jj0F4"
      }
      inputs {
        key: "0"
        value: "ref_PCollection_PCollection_1"
      }
      outputs {
        key: "None"
        value: "ref_PCollection_PCollection_2"
      }
      unique_name: "Sleep"
    }
  }
{noformat}

Additionally, the coders seem to be authored incorrectly, with a double nested spec:

{noformat}
 coders {
    key: "ref_Coder_VarIntCoder_1"
    value {
      spec {
        spec {
          urn: "beam:coder:varint:v1"
        }
      }
    }
  }
{noformat}

I have to remove one level of the spec to get it to read properly in Java.

To be clear, the issue that I'm describing occurs when I manually write and read the pipeline via protobufs: everything works fine when I submit using `pipe.run()`.  I surmise that there is some function in the python sdk that fixes up the pipeline message before sending it, otherwise I assume it _would_ be broken on receipt. Is that correct?





 

> Normalize URNs across SDKs and runners.
> ---------------------------------------
>
>                 Key: BEAM-3595
>                 URL: https://issues.apache.org/jira/browse/BEAM-3595
>             Project: Beam
>          Issue Type: Bug
>          Components: beam-model
>            Reporter: Robert Bradshaw
>            Assignee: Eugene Kirpichov
>            Priority: Major
>             Fix For: 2.5.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.14#76016)