[jira] [Commented] (BEAM-9615) [Go SDK] Beam Schemas

["Robert Burke (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-9615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233078#comment-17233078 ] 

Robert Burke commented on BEAM-9615:
------------------------------------

I now have Logical types working, and am working on being able to Provision protocol buffer types properly.  This wasn't something that was clear from what I read before writing the design doc.

> [Go SDK] Beam Schemas
> ---------------------
>
>                 Key: BEAM-9615
>                 URL: https://issues.apache.org/jira/browse/BEAM-9615
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-go
>            Reporter: Robert Burke
>            Assignee: Robert Burke
>            Priority: P2
>          Time Spent: 16h 50m
>  Remaining Estimate: 0h
>
> Schema support is required for advanced cross language features in Beam, and has the opportunity to replace the current default JSON encoding of elements.
> Some quick notes, though a better fleshed out doc with details will be forthcoming:
>  * All base coders should be implemented, and listed as coder capabilities. I think only stringutf8 is missing presently.
>  * Should support fairly arbitrary user types, seamlessly. That is, users should be able to rely on it "just working" if their type is compatible.
>  * Should support schema metadata tagging.
> In particular, one breaking shift in the default will be to explicitly fail pipelines if elements have unexported fields, when no other custom coder has been added. This has been a source of errors/dropped data/keys and a simply warning at construction time won't cut it. However, we could provide a manual "use beam schemas, but ignore unexported fields" registration as a work around.
> Edit: Doc is now at https://s.apache.org/beam-go-schemas



--
This message was sent by Atlassian Jira
(v8.3.4#803005)