You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@fory.apache.org by "silantev-ai (via GitHub)" <gi...@apache.org> on 2025/07/17 12:43:46 UTC
[I] [Question] Golang RegisterTagType array of structs [fory]
silantev-ai opened a new issue, #2423:
URL: https://github.com/apache/fory/issues/2423
### Question
version: https://pkg.go.dev/github.com/apache/fory/go/fory@v0.0.0-20250716053527-5a15097950c1?tab=versions
What is correct way to register a custom struct as array?
Following example is working fine until I fill Example.Items field with values
```golang
func example() {
// region struct
type Item struct {
Name string
}
type Example struct {
Items []Item
}
// endregion
//region init fory
fory := forygo.NewFory(true)
if err := fory.RegisterTagType("Example", Example{}); err != nil {
panic(err)
}
if err := fory.RegisterTagType("Item", Item{}); err != nil {
panic(err)
}
// endregion
// region example1
example1 := &Example{Items: []Item{}}
fmt.Println(example1)
bytes, _ := fory.Marshal(example1)
var deserialized interface{}
if err := fory.Unmarshal(bytes, &deserialized); err != nil {
panic(err)
}
fmt.Println(deserialized)
// endregion
// region example2
example2 := &Example{Items: []Item{{Name: "name"}}}
fmt.Println(example2)
bytes, _ = fory.Marshal(example2)
var deserialized2 interface{}
if err := fory.Unmarshal(bytes, &deserialized2); err != nil {
panic(err)
}
fmt.Println(deserialized2)
// endregion
}
```
```
panic: reflect: New(nil)
goroutine 13 [running]:
reflect.New({0x0, 0x0})
/Users/aleks.silantev/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.darwin-arm64/src/reflect/value.go:2998 +0xf0
github.com/apache/fory/go/fory.sliceSerializer.readDifferentTypes({{{0x1050811b0, 0x104e84460}, {0x0, 0x0, 0x0}, 0x140005a4930, 0x140005a4990, 0x0, 0x11, 0x0, ...}}, ...)
/Users/aleks.silantev/go/pkg/mod/github.com/apache/fory/go/fory@v0.0.0-20250716053527-5a15097950c1/slice.go:296 +0x268
github.com/apache/fory/go/fory.sliceSerializer.Read({{{0x1050811b0, 0x104e84460}, {0x0, 0x0, 0x0}, 0x140005a4930, 0x140005a4990, 0x0, 0x11, 0x0, ...}}, ...)
/Users/aleks.silantev/go/pkg/mod/github.com/apache/fory/go/fory@v0.0.0-20250716053527-5a15097950c1/slice.go:247 +0x598
github.com/apache/fory/go/fory.(*Fory).readData(0x140002360a0, 0x14000236280, {0x104d83f80, 0x140005ba270, 0x197}, {0x1050776f0, 0x140005c2070})
/Users/aleks.silantev/go/pkg/mod/github.com/apache/fory/go/fory@v0.0.0-20250716053527-5a15097950c1/fory.go:447 +0xb0
github.com/apache/fory/go/fory.(*Fory).readReferencableBySerializer(0x140002360a0, 0x14000236280, {0x104d83f80, 0x140005ba270, 0x197}, {0x1050776f0, 0x140005c2070})
/Users/aleks.silantev/go/pkg/mod/github.com/apache/fory/go/fory@v0.0.0-20250716053527-5a15097950c1/fory.go:390 +0x1e8
github.com/apache/fory/go/fory.readBySerializer(0x140002360a0, 0x14000236280, {0x104d83f80, 0x140005ba270, 0x197}, {0x1050776f0, 0x140005c2070}, 0x1)
/Users/aleks.silantev/go/pkg/mod/github.com/apache/fory/go/fory@v0.0.0-20250716053527-5a15097950c1/serializer.go:480 +0x6c
github.com/apache/fory/go/fory.(*structSerializer).Read(0x14000236150, 0x140002360a0, 0x14000236280, {0x1050811b0, 0x104e844e0}, {0x104e844e0, 0x140005ba270, 0x199})
/Users/aleks.silantev/go/pkg/mod/github.com/apache/fory/go/fory@v0.0.0-20250716053527-5a15097950c1/struct.go:111 +0x560
github.com/apache/fory/go/fory.(*ptrToStructSerializer).Read(0x14000236140, 0x140002360a0, 0x14000236280, {0x1050811b0, 0x104d6c980}, {0x104d6c980, 0x140005aa118, 0x196})
/Users/aleks.silantev/go/pkg/mod/github.com/apache/fory/go/fory@v0.0.0-20250716053527-5a15097950c1/struct.go:332 +0x128
github.com/apache/fory/go/fory.(*Fory).readData(0x140002360a0, 0x14000236280, {0x104e24220, 0x14000596440, 0x194}, {0x1050776b8, 0x14000236140})
/Users/aleks.silantev/go/pkg/mod/github.com/apache/fory/go/fory@v0.0.0-20250716053527-5a15097950c1/fory.go:441 +0x3a4
github.com/apache/fory/go/fory.(*Fory).readReferencableBySerializer(0x140002360a0, 0x14000236280, {0x104e24220, 0x14000596440, 0x194}, {0x0, 0x0})
/Users/aleks.silantev/go/pkg/mod/github.com/apache/fory/go/fory@v0.0.0-20250716053527-5a15097950c1/fory.go:390 +0x1e8
github.com/apache/fory/go/fory.(*Fory).ReadReferencable(0x140002360a0, 0x14000236280, {0x104e24220, 0x14000596440, 0x194})
/Users/aleks.silantev/go/pkg/mod/github.com/apache/fory/go/fory@v0.0.0-20250716053527-5a15097950c1/fory.go:377 +0x50
github.com/apache/fory/go/fory.(*Fory).Deserialize(0x140002360a0, 0x14000236280, {0x104d6f900, 0x14000596440}, {0x0, 0x0, 0x0})
/Users/aleks.silantev/go/pkg/mod/github.com/apache/fory/go/fory@v0.0.0-20250716053527-5a15097950c1/fory.go:370 +0x358
github.com/apache/fory/go/fory.(*Fory).Unmarshal(0x140002360a0, {0x1400059e150, 0x1f, 0x30}, {0x104d6f900, 0x14000596440})
/Users/aleks.silantev/go/pkg/mod/github.com/apache/fory/go/fory@v0.0.0-20250716053527-5a15097950c1/fory.go:326 +0xf0
```
Also It works fine with version:
github.com/alipay/fury/go/fury v0.0.0-20231216032808-080e351f29fb // https://pkg.go.dev/github.com/google/fury?tab=versions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [I] [Golang] RegisterTagType array of structs [fory]
Posted by "lzaeh (via GitHub)" <gi...@apache.org>.
lzaeh commented on issue #2423:
URL: https://github.com/apache/fory/issues/2423#issuecomment-3101463282
> Thank you for your explanation! I understand meaning of withRefTracking=true. Reducing the size over http is what I am trying to do. When I mentioned:
>
> > I am very interested in the flag withRefTracking=true, because it also does not work with XLANG mode, at least in the previous version.
>
> I meant it does not work properly in a cross language mode. In my case if I use Java for serialization, transfer data over http, and then use Golang for deserialization, will it work properly without any errors?
Personally, I wouldn’t recommend using the Go implementation for anything beyond what’s already covered in the test files — whether it's for Go itself or cross-language usage — because the current Go version is still not very mature.
In the version you were using before, there were some subtle issues when serializing and deserializing structs across languages. Some of those problems have been fixed in the current version, but there are still known bugs when it comes to arrays and slices — especially when you try to serialize or deserialize them on their own across languages. Within Go itself, array/slice-related issues are less common, but still something to be mindful of.
Also, when deserializing, I strongly recommend using the exact, correct types instead of falling back on interface{}. While the test files use interface{} for convenience, after some discussions we realized that in some cases it's actually important to use the precise type to receive the data correctly. That’s something I didn’t handle well earlier, and I plan to improve this part going forward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [I] [Golang] RegisterTagType array of structs [fory]
Posted by "pandalee99 (via GitHub)" <gi...@apache.org>.
pandalee99 commented on issue #2423:
URL: https://github.com/apache/fory/issues/2423#issuecomment-3121550797
@lzaeh Thank you very much for your support, this is a great idea!
I set up a process to keep track of fory-go's development.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [I] [Golang] RegisterTagType array of structs [fory]
Posted by "silantev-ai (via GitHub)" <gi...@apache.org>.
silantev-ai commented on issue #2423:
URL: https://github.com/apache/fory/issues/2423#issuecomment-3101521609
`it does fall under the known issues we currently have with arrays and slices.`
Do they(tickets) exist? Could you please leave the links to watch the progress on them?
There is no choice, I will wait for a stable version))
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [I] [Golang] RegisterTagType array of structs [fory]
Posted by "lzaeh (via GitHub)" <gi...@apache.org>.
lzaeh commented on issue #2423:
URL: https://github.com/apache/fory/issues/2423#issuecomment-3101478068
> Thank you for your explanation! I understand meaning of withRefTracking=true. Reducing the size over http is what I am trying to do. When I mentioned:
>
> > I am very interested in the flag withRefTracking=true, because it also does not work with XLANG mode, at least in the previous version.
>
> I meant it does not work properly in a cross language mode. In my case if I use Java for serialization, transfer data over http, and then use Golang for deserialization, will it work properly without any errors?
I'll work on fixing and implementing support for your specific case — it does fall under the known issues we currently have with arrays and slices. Really sorry for the inconvenience this has caused you. That said, all the examples in the test files, both for Go itself and for cross-language cases, should still work as expected for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [I] [Golang] RegisterTagType array of structs [fory]
Posted by "pandalee99 (via GitHub)" <gi...@apache.org>.
pandalee99 closed issue #2423: [Golang] RegisterTagType array of structs
URL: https://github.com/apache/fory/issues/2423
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [I] [Golang] RegisterTagType array of structs [fory]
Posted by "silantev-ai (via GitHub)" <gi...@apache.org>.
silantev-ai commented on issue #2423:
URL: https://github.com/apache/fory/issues/2423#issuecomment-3084353642
Thank you for quick reply! If it is an issue in the source code, could you please link this thread with a ticket? There is no way to upgrade the version until it is going to work properly. So yes I will stay on the previous version. By the way I use Java for serialization and Golang for deserialization. I am very interested in the flag `withRefTracking=true`, because it also does not work with `XLANG` mode, at least in the previous version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [I] [Golang] RegisterTagType array of structs [fory]
Posted by "lzaeh (via GitHub)" <gi...@apache.org>.
lzaeh commented on issue #2423:
URL: https://github.com/apache/fory/issues/2423#issuecomment-3101462620
Personally, I wouldn’t recommend using the Go implementation for anything beyond what’s already covered in the test files — whether it's for Go itself or cross-language usage — because the current Go version is still not very mature.
In the version you were using before, there were some subtle issues when serializing and deserializing structs across languages. Some of those problems have been fixed in the current version, but there are still known bugs when it comes to arrays and slices — especially when you try to serialize or deserialize them on their own across languages. Within Go itself, array/slice-related issues are less common, but still something to be mindful of.
Also, when deserializing, I strongly recommend using the exact, correct types instead of falling back on interface{}. While the test files use interface{} for convenience, after some discussions we realized that in some cases it's actually important to use the precise type to receive the data correctly. That’s something I didn’t handle well earlier, and I plan to improve this part going forward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [I] [Golang] RegisterTagType array of structs [fory]
Posted by "lzaeh (via GitHub)" <gi...@apache.org>.
lzaeh commented on issue #2423:
URL: https://github.com/apache/fory/issues/2423#issuecomment-3089961804
> Thank you for quick reply! If it is an issue in the source code, could you please link this thread with a ticket? There is no way to upgrade the version until it is going to work properly. So yes I will stay on the previous version. By the way I use Java for serialization and Golang for deserialization. I am very interested in the flag `withRefTracking=true`, because it also does not work with `XLANG` mode, at least in the previous version.
Hi, I’ve looked into the issue based on the code example you provided — thank you for spotting and sharing it.
In earlier versions where your code worked fine, cross-language support could still fail in certain cases. In the current version, some changes were introduced to fix what seemed to be a more critical issue, but they unfortunately introduced new problems — including the one you found.
I’ve summarized the remaining known issues here: https://github.com/apache/fory/pull/2427
If you need to work with the current version, you might be able to avoid these issues by steering clear of the affected patterns.
One thing worth mentioning: in your code sample you're using a slice, but in the description it's referred to as an array. In Go, slices and arrays are completely different types. Fortunately, the case in your code was easier to debug and fix. That said, support for slices and arrays is still not perfect in the current version — basic use cases work fine in Go itself, but cross-language compatibility still has problems.
I'm still new to the project and trying to work through these issues. Sorry for the trouble, and thank you again for your feedback and patience!
---
Regarding the withRefTracking option you mentioned — based on my understanding, Fory’s reference tracking mechanism naturally helps to:
Prevent infinite loops caused by cyclic references, and
Reduce the size of the serialized output in certain cases, which can help lower transmission overhead.
For example, consider the following two tests:
In the first test, with reference tracking enabled, the serialized output is significantly smaller in size. Without it, repeated strings are fully duplicated in the output, leading to much larger payloads.
<img width="887" height="335" alt="Image" src="https://github.com/user-attachments/assets/c5a2f2bc-8bce-4af8-a4ba-b48fc90101cc" />
<img width="718" height="266" alt="Image" src="https://github.com/user-attachments/assets/6fc7cc98-df05-4404-a921-e30465f10942" />
In the second test, we demonstrate a cyclic reference scenario.
The first screenshot shows that when reference tracking is disabled, Fory throws an error asking you to enable the option.
<img width="978" height="547" alt="Image" src="https://github.com/user-attachments/assets/d61dfe3a-fec4-4ff4-834b-ae91240e6664" />
<img width="1104" height="320" alt="Image" src="https://github.com/user-attachments/assets/144d5f20-cbf5-4b00-8416-7f78b96210ee" />
The second screenshot shows that serialization proceeds normally once reference tracking is turned on.
<img width="992" height="487" alt="Image" src="https://github.com/user-attachments/assets/466908fb-c451-4efc-a984-987d6f1e4b66" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [I] [Golang] RegisterTagType array of structs [fory]
Posted by "silantev-ai (via GitHub)" <gi...@apache.org>.
silantev-ai commented on issue #2423:
URL: https://github.com/apache/fory/issues/2423#issuecomment-3095593346
Thank you for your explanation! I understand meaning of withRefTracking=true. Reducing the size over http is what I am trying to do.
When I mentioned:
> I am very interested in the flag withRefTracking=true, because it also does not work with XLANG mode, at least in the previous version.
I meant it does not work properly in a cross language mode. In my case if I use Java for serialization, transfer data over http, and then use Golang for deserialization, will it work properly without any errors?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [I] [Golang] RegisterTagType array of structs [fory]
Posted by "lzaeh (via GitHub)" <gi...@apache.org>.
lzaeh commented on issue #2423:
URL: https://github.com/apache/fory/issues/2423#issuecomment-3101565693
> > it does fall under the known issues we currently have with arrays and slices.
>
> Do they(tickets) exist? Could you please leave the links to watch the progress on them? There is no choice, I will wait for a stable version))
I'll drop an email to one of the maintainers to check later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [I] [Golang] RegisterTagType array of structs [fory]
Posted by "lzaeh (via GitHub)" <gi...@apache.org>.
lzaeh commented on issue #2423:
URL: https://github.com/apache/fory/issues/2423#issuecomment-3084273856
Thank you so much for bringing this to our attention. I sincerely apologize for the issues you’ve encountered with array and slice handling in the latest release.
In previous versions, we relied on a somewhat brittle mechanism to power our serialization and deserialization, and in an effort to enable true cross‑language support, I attempted to remove it—but clearly I didn’t cover all the cases. You’re absolutely right that array and slice types aren’t yet fully supported in this new implementation.
I’m very sorry for the inconvenience this has caused. In the meantime, please feel free to continue using the prior version, and I will prioritize analyzing and resolving this problem as quickly as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org