[jira] [Assigned] (BEAM-11095) Improve error messages from Invalid element types in emitter signatures

["Rehman Murad Ali (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-11095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rehman Murad Ali reassigned BEAM-11095:
---------------------------------------

    Assignee: Rehman Murad Ali

> Improve error messages from Invalid element types in emitter signatures
> -----------------------------------------------------------------------
>
>                 Key: BEAM-11095
>                 URL: https://issues.apache.org/jira/browse/BEAM-11095
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-go
>            Reporter: Robert Burke
>            Assignee: Rehman Murad Ali
>            Priority: P3
>              Labels: starter
>
> If a DoFn's `ProcessElement`uses an emitter `func(MyType)` where `MyType` is not a concrete type, (such as having function type fields, which are not valid) the pipeline can fail with incomprehensible errors that don't explain the problem correctly. eg.
> {{panic: inserting ParDo in scope *mypackage.studyGenerationElement
>  constructing DoFn
> invalid DoFn
>  caused by:
> method ProcessElement invalid
>  caused by:
> bad parameter type for reflect.methodValueCall: func(*mypackage.studyGenerationElement)
> goroutine 740 [running]:
> panic(0x55e4c34f3e80, 0xc005952680)
> go/gc/src/runtime/panic.go:1066+0x47e fp=0xc0061a18c8 sp=0xc0061a17d8 pc=0x55e49ff4b67e
> testing.tRunner.func1.1(0x55e4c34f3e80, 0xc005952680)
> go/gc/src/testing/testing.go:1076 +0x30d fp=0xc0061a1980 sp=0xc0061a18c8 pc=0x55e4a097254d
> testing.tRunner.func1(0xc005046900)
> go/gc/src/testing/testing.go:1079 +0x41a fp=0xc0061a1a28 sp=0xc0061a1980 pc=0x55e4a097297a
> runtime.call32(0x0, 0x55e4c3f0e388, 0xc0050dbf38, 0x800000008)
> go/gc/src/runtime/asm_amd64.s:540 +0x40 fp=0xc0061a1a58 sp=0xc0061a1a28 pc=0x55e49ff86fc0
> runtime.reflectcallSave(0xc0061a1b88, 0x55e4c3f0e388, 0xc0050dbf38, 0x55e400000008)
> go/gc/src/runtime/panic.go:881 +0x5a fp=0xc0061a1a88 sp=0xc0061a1a58 pc=0x55e49ff4b13a
> runtime.runOpenDeferFrame(0xc0035bc180, 0xc0050dbef0, 0xc0061a1c08)
> go/gc/src/runtime/panic.go:855 +0x2cd fp=0xc0061a1b18 sp=0xc0061a1a88 pc=0x55e49ff4afed
> panic(0x55e4c34f3e80, 0xc005952680)
> go/gc/src/runtime/panic.go:969 +0x175 fp=0xc0061a1c08 sp=0xc0061a1b18 pc=0x55e49ff4b375
> ...pkg/beam/beam.MustN(...)...pkg/beam/util.go:104
> ...pkg/beam/beam.ParDo(0xc00507d9e0, 0xc0052a7b30, 0x55e4c30faae0, 0xc0053244e0, 0xc0053245a0, 0x0, 0x0, 0x0, 0x0)
> ...pkg/beam/pardo.go:358 +0x135 fp=0xc0061a1c80 sp=0xc0061a1c08 pc=0x55e4a0f3cc95
> }}
> In particular this message is generated by [funcx.New|https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/funcx/fn.go#L323] and occurs because we don't maintain an error state when trying to analyze the DoFn.
> We would catch most errors like this if we provide a justification for it better.
> This will likely mean reimplementing the IsConcrete function and similar in terms of errors with the explanations, and having a justification handler to return that error from funcx.New if the type is invalid. This is subtle as while most errors are due to a type not being concrete, but the type might be nested in a emitter or iterator function.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)