[jira] [Commented] (BEAM-5298) Determine what painpoints exist for the Go SDK WRT the Go Generics Draft Design

["Robert Burke (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-5298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16746566#comment-16746566 ] 

Robert Burke commented on BEAM-5298:
------------------------------------

I'm finally in a place where it's worth putting thought into this, but just a comment based on recent discussion from the dev list:

With the discussion of Schemas and User Coders, it's come to my attention that the *typed generic* PCollections that Java has are becoming a bit of an inconvenience. With typed PCollections, essentially *everything* must be a PCollection<Schema>, or it would fail at compile time. However, a PCollection<Schema> has the opportunity in Go to be treated as any compatible concrete type by the user code, mostly because we already do Construction Time type checking of a pipeline graph.  In short, if Beam moves towards broad inter-language schema support, then even with Generics in Go, the PCollections will remain untyped, and we'd retain the construction time type checking as we do now.

> Determine what painpoints exist for the Go SDK WRT the Go Generics Draft Design
> -------------------------------------------------------------------------------
>
>                 Key: BEAM-5298
>                 URL: https://issues.apache.org/jira/browse/BEAM-5298
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-go
>            Reporter: Robert Burke
>            Assignee: Robert Burke
>            Priority: Major
>
> Specifically, the outcome should be a doc outlining what would need to change in the SDK as a consequence of the draft design, with a focus on the user experience, and how much unnecessary user toil could be avoided.
> [https://go.googlesource.com/proposal/+/master/design/go2draft-generics-overview.md]
> Initial impressions:
> As the draft design indicates, the current SDK code would not be required to change, it would continue working as is, as the change is intended to be backwards compatible. Users should be able to use their own generic code with the v1 of the SDK.
>  
> A v2 of the Go SDK could remove much of the code the SDK spends in doing it's own type checking of pipelines at construction time, in favour of the compiler.
> PCollections would be properly typed to facilitate this.
> Specific focus should be made to cover the user experience side, and sketch implementations of the generic types (like PCollection) and functions (like ParDo), as well as on the execution harness side. eg. is it possible to avoid reflection or type assertion on the processing path?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)