You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/09/08 17:27:13 UTC

[GitHub] [beam] lostluck opened a new issue, #23101: [Feature Request]:[Go SDK]: Improve error message for unencodeable types w/ Workarounds

lostluck opened a new issue, #23101:
URL: https://github.com/apache/beam/issues/23101

   ### What would you like to happen?
   
   Map types are able to be encoded if they are fields in a struct. This lets them take advantage of the Beam Schema Row coder which knows how to encode these fields.
   
   For maps in particular, we should suggest the work around of wrapping in an explicit struct type with an exported field.
   Slices and Arrays may need a similar treatment.
   
   Channels, funcs, and unregistered interface types remain unencodeable.
   ```
   F0907 15:34:11.803229 3396182 main.go:87] Failed to execute job: failed to generate model pipeline: failed to encode custom coder for type json. Make sure the type was registered before calling beam.Init. For example: beam.RegisterType(reflect.TypeOf((*TypeName)(nil)).Elem())
   Full error:
   failed to add input kind: {google3/third_party/golang/apache_beam/pkg/beam/beam.dropKeyFn 8: ParDo [In(Main): KV<X,Y> <- {7: KV<int,map[string]int>/KV<int[varintz],map[string]int[json]> GLO}] -> [Out: Y -> {8: map[string]int/map[string]int[json] GLO}]}
   	caused by:
   failed to marshal KV coder KV<int[varintz],map[string]int[json]>
   	caused by:
   failed to marshal the coder map[string]int[json].
   	caused by:
   	encoding custom coder map[string]int[json] for type map[string]int
   unencodable type map[string]int
   ```
   
   ### Issue Priority
   
   Priority: 3
   
   ### Issue Component
   
   Component: sdk-go


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] Johnny11502 commented on issue #23101: [Feature Request]:[Go SDK]: Improve error message for unencodeable types w/ Workarounds

Posted by GitBox <gi...@apache.org>.
Johnny11502 commented on issue #23101:
URL: https://github.com/apache/beam/issues/23101#issuecomment-1275270639

   Hello @lostluck , I am a student learning the ins and outs of GitHub and currently have an assignment that requires us to have an issue that we feel confident in solving assigned to us. I would appreciate it if you could assign this issue to me so I can give it a try. Thanks in advance!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] Johnny11502 commented on issue #23101: [Feature Request]:[Go SDK]: Improve error message for unencodeable types w/ Workarounds

Posted by GitBox <gi...@apache.org>.
Johnny11502 commented on issue #23101:
URL: https://github.com/apache/beam/issues/23101#issuecomment-1275450917

   Got it. Thanks for the information!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] Johnny11502 commented on issue #23101: [Feature Request]:[Go SDK]: Improve error message for unencodeable types w/ Workarounds

Posted by GitBox <gi...@apache.org>.
Johnny11502 commented on issue #23101:
URL: https://github.com/apache/beam/issues/23101#issuecomment-1289347702

   @lostluck  Sorry for the lack of updates but I did end up completing a different issue for my assignment. This was one of the three I chose in case I had issues getting the first one assigned to me. I appreciate you allowing me to keep it for so long, Thank you.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] yyy1000 commented on issue #23101: [Feature Request]:[Go SDK]: Improve error message for unencodeable types w/ Workarounds

Posted by GitBox <gi...@apache.org>.
yyy1000 commented on issue #23101:
URL: https://github.com/apache/beam/issues/23101#issuecomment-1365779041

   I reproduced the error message successfully. 
   But when I use the struct type to wrap the Map type, there's still a problem. It seems the map struct is still unencodable.
   Could you please help to see what's my problem?  :)
   My code is like this:
   ```
   type MyNewStruct struct {
   	Map map[int]int
   }
   func Test(t *testing.T) {
   	f, _ := coderx.NewVarIntZ(reflectx.Int)
   	foo := &coder.Coder{Kind: coder.Custom, T: typex.New(reflectx.Int), Custom: f}
           //coder.NewCustomCoder
   	bar := custom("bar", reflect.TypeOf((*MyNewStruct)(nil)).Elem())
   	tests := []struct {
   		name       string
   		c          *coder.Coder
   		equivalent *coder.Coder
   	}{
   		{
   			name: "KV<foo,bar>",
   			c:    coder.NewKV([]*coder.Coder{foo,bar}),
   		},
   	}
   
   	for _, test := range tests {
   		t.Run(test.name, func(t *testing.T) {
   			_, _, err := graphx.MarshalCoders([]*coder.Coder{test.c})
   			if err != nil {
   				t.Fatalf("Marshal(%v) failed: %v", test.c, err)
   			}
   		})
   	}
   	}
   ```
   The error message is:
   
   > Full error:
               failed to marshal the coder KV<int[varintz],graphx_test.MyNewStruct[bar]>.
               	caused by:
               failed to marshal KV coder KV<int[varintz],graphx_test.MyNewStruct[bar]>
               	caused by:
               failed to marshal the coder graphx_test.MyNewStruct[bar].
               	caused by:
               	encoding custom coder graphx_test.MyNewStruct[bar] for type graphx_test.MyNewStruct
               	encoding struct graphx_test.MyNewStruct
               bad field type
               	caused by:
               unencodable type map[int]int


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on issue #23101: [Feature Request]:[Go SDK]: Improve error message for unencodeable types w/ Workarounds

Posted by GitBox <gi...@apache.org>.
lostluck commented on issue #23101:
URL: https://github.com/apache/beam/issues/23101#issuecomment-1275431369

   That page should also have some getting started contribution information (like cloning the repo, etc). Remember, if you read all the documentation, and it doesn't work as described, it's just as likely the documentation is wrong, as you've made a typo or similar. 
   
   So please do file an issue (or a PR to fix the documentation!) if you run into something like that.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck closed issue #23101: [Feature Request]:[Go SDK]: Improve error message for unencodeable types w/ Workarounds

Posted by GitBox <gi...@apache.org>.
lostluck closed issue #23101: [Feature Request]:[Go SDK]: Improve error message for unencodeable types w/ Workarounds
URL: https://github.com/apache/beam/issues/23101


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on issue #23101: [Feature Request]:[Go SDK]: Improve error message for unencodeable types w/ Workarounds

Posted by GitBox <gi...@apache.org>.
lostluck commented on issue #23101:
URL: https://github.com/apache/beam/issues/23101#issuecomment-1364759870

   An improved error message is something that's clearer to someone who isn't intimately familiar with the SDK. To me forexample, it's pretty clear, but I also know what it's talking about. I'm also unlikely to make mistakes in pipeline construction that would lead to the error.
   
   Here's my suggestion: look at the current error message (see the original description) and figure out what's good about it, and what's bad about it. Then fix the bad parts about it, by adding more information, or reformating the information.
   
   A good error message is one that clearly indicates what's wrong, ideally in terms the user understands, or in terms of what the user specified, describes why that's wrong, and finally suggests a way to fix it. In this case, that would be the work around(s).
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on issue #23101: [Feature Request]:[Go SDK]: Improve error message for unencodeable types w/ Workarounds

Posted by GitBox <gi...@apache.org>.
lostluck commented on issue #23101:
URL: https://github.com/apache/beam/issues/23101#issuecomment-1275428686

   Sure! Let me know if you have any questions.
   
   For future reference, per beam.apache.org/contribute , you can comment  “.take-issue”  and a bot will make sure you're assigned. :) 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] yyy1000 commented on issue #23101: [Feature Request]:[Go SDK]: Improve error message for unencodeable types w/ Workarounds

Posted by GitBox <gi...@apache.org>.
yyy1000 commented on issue #23101:
URL: https://github.com/apache/beam/issues/23101#issuecomment-1363833855

   I could work for this.
   What's the possible improved result for the error message? 
   Could you please describe that in detail? Thanks! :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] yyy1000 commented on issue #23101: [Feature Request]:[Go SDK]: Improve error message for unencodeable types w/ Workarounds

Posted by GitBox <gi...@apache.org>.
yyy1000 commented on issue #23101:
URL: https://github.com/apache/beam/issues/23101#issuecomment-1363833997

   .take-issue


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org