You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/10/12 08:30:23 UTC
[GitHub] [pulsar-client-go] YanshuoH opened a new issue, #862: gogo/protobuf is deprecated
YanshuoH opened a new issue, #862:
URL: https://github.com/apache/pulsar-client-go/issues/862
**Is your feature request related to a problem? Please describe.**
Nowadays, the famous https://github.com/gogo/protobuf is deprecated and it is not compatible with all brand new golang protobuf features.
For instance, when using protobuf as schema, it prompts some errors like such:
```
Test Panicked
protobuf tag not enough fields in Record.state:
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/protobuf@v1.3.2/proto/table_unmarshal.go:341
```
<details>
<summary>click to see the full logs</summary>
```
Schema
/Users/hys/dev/go-workspace/src/go.planetmeican.com/nerds/mq/pulsar/schema_test.go:72
ProduceAndConsume
/Users/hys/dev/go-workspace/src/go.planetmeican.com/nerds/mq/pulsar/schema_test.go:92
with avro and json and protobuf schema
/Users/hys/dev/go-workspace/src/go.planetmeican.com/nerds/mq/pulsar/schema_test.go:100
should be able to consume the produced message [It]
/Users/hys/dev/go-workspace/src/go.planetmeican.com/nerds/mq/pulsar/schema_test.go:101
Test Panicked
protobuf tag not enough fields in Record.state:
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/protobuf@v1.3.2/proto/table_unmarshal.go:341
Full Stack Trace
github.com/gogo/protobuf/proto.(*unmarshalInfo).computeUnmarshalInfo(0xc0000ac000)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/protobuf@v1.3.2/proto/table_unmarshal.go:341 +0x138a
github.com/gogo/protobuf/proto.(*unmarshalInfo).unmarshal(0xc0000ac000, {0x489e980}, {0xc00034c0de, 0x29, 0x29})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/protobuf@v1.3.2/proto/table_unmarshal.go:138 +0x67
github.com/gogo/protobuf/proto.(*InternalMessageInfo).Unmarshal(0xc000436680, {0x2d7d7a40, 0xc0002ed740}, {0xc00034c0de, 0x29, 0x29})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/protobuf@v1.3.2/proto/table_unmarshal.go:63 +0xd0
github.com/gogo/protobuf/proto.(*Buffer).Unmarshal(0xc00081ec78, {0x2d7d7a40, 0xc0002ed740})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/protobuf@v1.3.2/proto/decode.go:424 +0x153
github.com/gogo/protobuf/proto.Unmarshal({0xc00034c0de, 0x29, 0x29}, {0x2d7d7a40, 0xc0002ed740})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/protobuf@v1.3.2/proto/decode.go:342 +0xef
github.com/apache/pulsar-client-go/pulsar.(*ProtoSchema).Decode(0x6900f18, {0xc00034c0de, 0x29, 0x29}, {0x48896a0, 0xc0002ed740})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/apache/pulsar-client-go@v0.8.1/pulsar/schema.go:149 +0x58
github.com/apache/pulsar-client-go/pulsar.(*message).GetSchemaValue(0x58, {0x48896a0, 0xc0002ed740})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/apache/pulsar-client-go@v0.8.1/pulsar/impl_message.go:291 +0x4d
go.planetmeican.com/nerds/mq/pulsar_test.glob..func8.3.1.1()
/Users/hys/dev/go-workspace/src/go.planetmeican.com/nerds/mq/pulsar/schema_test.go:200 +0x134c
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync(0xc00081f440)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/ginkgo@v1.14.2/internal/leafnodes/runner.go:113 +0xba
github.com/onsi/ginkgo/internal/leafnodes.(*runner).run(0x0)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/ginkgo@v1.14.2/internal/leafnodes/runner.go:64 +0x125
github.com/onsi/ginkgo/internal/leafnodes.(*ItNode).Run(0xc00081f760)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/ginkgo@v1.14.2/internal/leafnodes/it_node.go:26 +0x7b
github.com/onsi/ginkgo/internal/spec.(*Spec).runSample(0xc00027fb30, 0xc00081faa8, {0x49f03c0, 0xc000450180})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/ginkgo@v1.14.2/internal/spec/spec.go:215 +0x2a9
github.com/onsi/ginkgo/internal/spec.(*Spec).Run(0xc00027fb30, {0x49f03c0, 0xc000450180})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/ginkgo@v1.14.2/internal/spec/spec.go:138 +0xe7
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpec(0xc0001fe280, 0xc00027fb30)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/ginkgo@v1.14.2/internal/specrunner/spec_runner.go:200 +0xe5
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpecs(0xc0001fe280)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/ginkgo@v1.14.2/internal/specrunner/spec_runner.go:170 +0x1a5
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run(0xc0001fe280)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/ginkgo@v1.14.2/internal/specrunner/spec_runner.go:66 +0xc5
github.com/onsi/ginkgo/internal/suite.(*Suite).Run(0xc00043a2a0, {0x2d8ad520, 0xc000102b60}, {0x48e4ebe, 0x4d27b20}, {0xc000118030, 0x1, 0x1}, {0x4a06f58, 0xc000450180}, ...)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/ginkgo@v1.14.2/internal/suite/suite.go:79 +0x4d2
github.com/onsi/ginkgo.RunSpecsWithCustomReporters({0x49f1180, 0xc000102b60}, {0x48e4ebe, 0xc}, {0xc00013b718, 0x1, 0xc00013b728})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/ginkgo@v1.14.2/ginkgo_dsl.go:229 +0x185
github.com/onsi/ginkgo.RunSpecs({0x49f1180, 0xc000102b60}, {0x48e4ebe, 0xc})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/ginkgo@v1.14.2/ginkgo_dsl.go:210 +0x10b
go.planetmeican.com/nerds/mq/pulsar_test.TestPulsar(0x0)
/Users/hys/dev/go-workspace/src/go.planetmeican.com/nerds/mq/pulsar/pulsar_suite_test.go:23 +0x85
testing.tRunner(0xc000102b60, 0x4921b20)
/Users/hys/.gvm/gos/go1.17/src/testing/testing.go:1259 +0x102
created by testing.(*T).Run
/Users/hys/.gvm/gos/go1.17/src/testing/testing.go:1306 +0x35a
```
</details>
Furthermore, using the `google.golang.org/protobuf/reflect/protoreflect` and the `google.golang.org/protobuf/types/dynamicpb`
to create dynamic protobuf message also causes trouble.
For example, `gogo/protobuf` has the checking tags issue: https://github.com/gogo/protobuf/issues/678
```
protobuf tag not enough fields in Message.typ:
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/protobuf@v1.3.2/proto/table_unmarshal.go:341
```
<details>
<summary>Example code to create dynamic object</summary>
```
getProtoMsgDescriptor := func() protoreflect.MessageDescriptor {
fddef := &descriptorpb.FileDescriptorProto{
Name: proto.String("test.proto"),
Package: proto.String("test"),
MessageType: []*descriptorpb.DescriptorProto{
{
Name: proto.String("Record"),
Field: []*descriptorpb.FieldDescriptorProto{
{
Name: proto.String("snowflake_id"),
JsonName: proto.String("snowflake_id,omitempty"),
Number: proto.Int32(1),
Type: descriptorpb.FieldDescriptorProto_Type(protoreflect.StringKind).Enum(),
},
{
Name: proto.String("occur_timestamp"),
JsonName: proto.String("occur_timestamp,omitempty"),
Number: proto.Int32(2),
Type: descriptorpb.FieldDescriptorProto_Type(protoreflect.Int64Kind).Enum(),
},
{
// usually we use google.protobuf.StringValue to do nullable string,
// but use dynamic message it is too much painful to include it into the resolver scope.
// so we'll instead use plain string.
Name: proto.String("from_system"),
JsonName: proto.String("from_system,omitempty"),
Number: proto.Int32(3),
Type: descriptorpb.FieldDescriptorProto_Type(protoreflect.StringKind).Enum(),
},
{
Name: proto.String("action_type"),
JsonName: proto.String("action_type,omitempty"),
Number: proto.Int32(4),
Type: descriptorpb.FieldDescriptorProto_Type(protoreflect.StringKind).Enum(),
},
},
},
},
Syntax: proto.String("proto3"),
}
fd, err := protodesc.NewFile(fddef, nil)
Expect(err).NotTo(HaveOccurred(), "%+v", err)
md := fd.Messages().ByName("Record")
return md
}
```
</details>
**Describe the solution you'd like**
My suggestion would be to replace all `gogo/protobuf` usage by the official `google.golang.org/protobuf`.
Per my understanding, this may require a re-generation of all generated pb.go files and some imports.
Thanks for your attention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] nodece closed issue #862: gogo/protobuf is deprecated
Posted by GitBox <gi...@apache.org>.
nodece closed issue #862: gogo/protobuf is deprecated
URL: https://github.com/apache/pulsar-client-go/issues/862
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] nodece commented on issue #862: gogo/protobuf is deprecated
Posted by GitBox <gi...@apache.org>.
nodece commented on issue #862:
URL: https://github.com/apache/pulsar-client-go/issues/862#issuecomment-1331717701
Good! Could you help to review #895?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org