[jira] [Created] (BEAM-13420) [Go SDK] Decoding a schema row doesn't respect field names from struct tags

["Daniel Oliveira (Jira)" <ji...@apache.org>]

Daniel Oliveira created BEAM-13420:
--------------------------------------

             Summary: [Go SDK] Decoding a schema row doesn't respect field names from struct tags
                 Key: BEAM-13420
                 URL: https://issues.apache.org/jira/browse/BEAM-13420
             Project: Beam
          Issue Type: Bug
          Components: sdk-go
            Reporter: Daniel Oliveira


Will attempt to create a reproducible code snippet soon.

For now, the basics of the bug are that when an element encoded as a Row via schemas gets decoded to a Go struct, it doesn't respect struct tags somewhere in the conversion process. More specifically, if I have an external transform that outputs a Row, and I take that as input to a native Go transform that accepts some Go struct "Foo", it will fail if the field names are different, even if the struct tags on Foo match the external row's field names.

Example error message. The following error is caused by the WordCount and CorpusDate fields in the native struct not matching the field names "word_count" and "corpus_date" from the raw Row. The row gets decoded into a struct with the field names Word_count and Corpus_date, and ignoring the struct tags of the struct it's attempting to match:
{noformat}
panic: reflect: Call using struct { Word string "beam:\"word\""; Word_count int64 "beam:\"word_count\""; Corpus string "beam:\"corpus\""; Corpus_date int64 "beam:\"corpus_date\"" } as type struct { Word string "beam:\"word\""; WordCount int64 "beam:\"word_count\""; Corpus string "beam:\"corpus\""; CorpusDate int64 "beam:\"corpus_date\"" }
Full error:
while executing Process for Plan[process-bundle-descriptor-291]:
2: DataSink[S[ptransform-289@localhost:12371]] Coder:W;coder-315<KV;coder-316<int[varintz;c3];coder-317,string;coder-318>>!GWC
3: PCollection[pcollection-304] Out:[2]
4: ParDo[beam.addFixedKeyFn] Out:[2]
5: PCollection[pcollection-300] Out:[4]
6: ParDo[main.main.func1] Out:[5]
1: DataSource[S[ptransform-288@localhost:12371], 0] Coder:W;coder-310<LP;coder-311<R[struct { Word string "beam:\"word\""; Word_count int64 "beam:\"word_count\""; Corpus string "beam:\"corpus\""; Corpus_date int64 "beam:\"corpus_date\"" }]>>!GWC Out:6
	caused by:
panic: reflect: Call using struct { Word string "beam:\"word\""; Word_count int64 "beam:\"word_count\""; Corpus string "beam:\"corpus\""; Corpus_date int64 "beam:\"corpus_date\"" } as type struct { Word string "beam:\"word\""; WordCount int64 "beam:\"word_count\""; Corpus string "beam:\"corpus\""; CorpusDate int64 "beam:\"corpus_date\"" } goroutine 39 [running]:
runtime/debug.Stack()
	/usr/lib/google-golang/src/runtime/debug/stack.go:24 +0x65
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.callNoPanic.func1()
	/usr/local/google/home/danoliveira/repos/beam/sdks/go/pkg/beam/core/runtime/exec/util.go:58 +0xa5
panic({0xe8f160, 0xc0003149e0})
	/usr/lib/google-golang/src/runtime/panic.go:1038 +0x215
reflect.Value.call({0xc0003267e0, 0xc0003b3e10, 0x5be732}, {0x102e0e5, 0x4}, {0xc0000fe3d8, 0x1, 0x679031})
	/usr/lib/google-golang/src/reflect/value.go:411 +0x1965
reflect.Value.Call({0xc0003267e0, 0xc0003b3e10, 0x1}, {0xc0000fe3d8, 0x1, 0x1})
	/usr/lib/google-golang/src/reflect/value.go:339 +0xc5
github.com/apache/beam/sdks/v2/go/pkg/beam/core/util/reflectx.(*reflectFunc).Call(0xc000426060, {0xc0003326f0, 0x0, 0xf5ef00})
	/usr/local/google/home/danoliveira/repos/beam/sdks/go/pkg/beam/core/util/reflectx/call.go:87 +0x59
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*invoker).initCall.func33({0x18e7a90, 0x1, 0x1}, 0xffdf3b645a1cac09)
	/usr/local/google/home/danoliveira/repos/beam/sdks/go/pkg/beam/core/runtime/exec/fn_arity.go:229 +0x7b
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*invoker).Invoke(0xc000477860, {0x1168a50, 0xc00033cdc0}, {0x18e7a90, 0x0, 0x1}, 0x203000, 0xc00015d660, {0x1934830, 0x0, ...})
	/usr/local/google/home/danoliveira/repos/beam/sdks/go/pkg/beam/core/runtime/exec/fn.go:186 +0x7a2
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*ParDo).invokeProcessFn(0xc0004420e0, {0x1168a50, 0xc00033cdc0}, {0x18e7a90, 0x1, 0x1}, 0x30, 0x1904be0)
	/usr/local/google/home/danoliveira/repos/beam/sdks/go/pkg/beam/core/runtime/exec/pardo.go:316 +0x146
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*ParDo).processSingleWindow(0xc0004420e0, 0xc00015d660)
	/usr/local/google/home/danoliveira/repos/beam/sdks/go/pkg/beam/core/runtime/exec/pardo.go:166 +0x4b
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*ParDo).processMainInput(0xc000348fc0, 0x1168a50)
	/usr/local/google/home/danoliveira/repos/beam/sdks/go/pkg/beam/core/runtime/exec/pardo.go:146 +0x9c
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*ParDo).ProcessElement(0xc0004420e0, {0x114d8a0, 0xc000426300}, 0xc0001f2540, {0x0, 0x0, 0x0})
	/usr/local/google/home/danoliveira/repos/beam/sdks/go/pkg/beam/core/runtime/exec/pardo.go:132 +0x1a5
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*DataSource).Process(0xc0000ec8c0, {0x1168a50, 0xc00033cd00})
	/usr/local/google/home/danoliveira/repos/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go:174 +0x4ab
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.callNoPanic({0x1168a50, 0xc00033cd00}, 0xc0002d6240)
	/usr/local/google/home/danoliveira/repos/beam/sdks/go/pkg/beam/core/runtime/exec/util.go:62 +0x6c
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*Plan).Execute(0xc000457030, {0x1168a50, 0xc00033cd00}, {0xc0003f07b0, 0x24}, {{0x115ee60, 0xc0002d61b0}, {0x11631d8, 0xc0002d61e0}})
	/usr/local/google/home/danoliveira/repos/beam/sdks/go/pkg/beam/core/runtime/exec/plan.go:118 +0x3a5
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/harness.(*control).handleInstruction(0xc000452000, {0x1168938, 0xc00033e0f0}, 0xc0003fe550)
	/usr/local/google/home/danoliveira/repos/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go:331 +0xb6f
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/harness.Main.func3({0x1168938, 0xc00033e0f0}, 0xc0003fe550)
	/usr/local/google/home/danoliveira/repos/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go:144 +0x19d
created by github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/harness.Main
	/usr/local/google/home/danoliveira/repos/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go:163 +0xa8f{noformat}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)