[jira] [Commented] (BEAM-3612) Make it easy to generate type-specialized Go SDK reflectx.Funcs

["Robert Burke (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-3612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16637189#comment-16637189 ] 

Robert Burke commented on BEAM-3612:
------------------------------------

I've got a prototype tool I've been working on to simplify this.

It should be able to replace the necessity of the optimized package. There are two avenues I'm exploring.



First: A vetting/verifying runner that can detect if a pipeline is "optimized" via avoiding the default reflective paths. This will largely check if all the various types and functions are registered with beam, as well as each having the appropriate type assertion shims registered as well. Optionally it can also generate a file to be included in the pipeline's package to optimize any types and functions missing that are using exported types and functions. In principle, this could be combined with the "compile for the execution architecture" approach the universal runner takes

Second: A static analysis based approach for library authors, that could be used with go generate. Having a package centric approach has the benefit allowing library authors to produce performant transforms for pipeline authors to use. It also avoids redundant type and function registrations. The main redundancy is that the reflectx.Funcs might be regenerated and registered if many packages provide the same signatures. In practice this is not likely a huge concern, except for binary bloat. On the whole though, I suspect not having all the speculative registrations of the optimized package will offset this.

Other than submitting the tools I've got once they're ready, the main change to the beam APIs is providing a way for the vetting/verifying runner to see if Iterators and Emitters have type specializations registered, as the exec package provides no way to examine the registry's contents at this time either by invoking the maker/wrapper functions or by an explicit IsRegistered function for each. These will need to be added.

> Make it easy to generate type-specialized Go SDK reflectx.Funcs
> ---------------------------------------------------------------
>
>                 Key: BEAM-3612
>                 URL: https://issues.apache.org/jira/browse/BEAM-3612
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-go
>            Reporter: Henning Rohde
>            Assignee: Robert Burke
>            Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)