[GitHub] [beam] lostluck commented on a change in pull request #10991: [BEAM-3301] Refactor DoFn validation & allow specifying main inputs.

[GitBox <gi...@apache.org>]

lostluck commented on a change in pull request #10991: [BEAM-3301] Refactor DoFn validation & allow specifying main inputs.
URL: https://github.com/apache/beam/pull/10991#discussion_r385468257
 
 

 ##########
 File path: sdks/go/pkg/beam/core/graph/fn.go
 ##########
 @@ -209,21 +209,58 @@ func (f *DoFn) RestrictionT() *reflect.Type {
 // a KV or not based on the other signatures (unless we're more loose about which
 // sideinputs are present). Bind should respect that.
 
+// Constants so we can avoid magic numbers in validation. Represent number of
+// DoFn main inputs based on what kind of input the DoFn has.
+const (
+	unknownInNum = -1 // Used when we don't know the number of main inputs.
+	singleInNum  = 1
+	kvInNum      = 2
+)
+
 // NewDoFn constructs a DoFn from the given value, if possible.
 func NewDoFn(fn interface{}) (*DoFn, error) {
 	ret, err := NewFn(fn)
 	if err != nil {
 		return nil, errors.WithContext(errors.Wrapf(err, "invalid DoFn"), "constructing DoFn")
 	}
-	return AsDoFn(ret)
+	return AsDoFn(ret, unknownInNum)
 }
 
-// AsDoFn converts a Fn to a DoFn, if possible.
-func AsDoFn(fn *Fn) (*DoFn, error) {
+// NewDoFnKv constructs a DoFn from the given value, if possible, with
+// improved validation from knowing whether the DoFn's main input is a KV or
+// single element.
+func NewDoFnKv(fn interface{}, mainKv bool) (*DoFn, error) {
 
 Review comment:
   With the name NewDoFnKv, it sounds like it's already assuming that a DoFn KV is being passed in. It's OK for there to be special purpose methods that only do one thing.
   
   Another option to consider instead of having two (or N) methods, consider extending the current NewDoFn with a variadic an Option type.  (eg. opts ...Option), this lets existing callers keep things the same, but allow for expanding things in the future. Option should probably be either a function type, or an interface type with private methods, and the options are provided by other methods in the package. eg. graph.NewDoFn(fn, graph.HasKVInput(), graph.HasRestriction()). This is valuable if we think being able to expand things in the future, but also lets us mix and match more easily later on. This way we can keep the existing behavior when there are no options, but keep the documentation of all the various uses in one place on the NewDoFn method referring to the option returning methods.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services