You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2021/09/23 10:15:41 UTC
[GitHub] [rocketmq] liuxuzxx opened a new issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
liuxuzxx opened a new issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369
**BUG REPORT**
When I use rocketmq-client-go fetch cluster information ,then i get the json data is:
```json
{"brokerAddrTable":{"broker_37_master":{"brokerAddrs":{0:"172.16.16.37:10911"},"brokerName":"broker_37_master","cluster":"37_cluster"}},"clusterAddrTable":{"37_cluster":["broker_37_master"]}}
```
then the golang json parse occure error:
```golang
err := json.Unmarshal(response, &temp)
```
and the error information is:
```
invalid character '0' looking for beginning of object key string
```
but JSON key must be string!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] liuxuzxx edited a comment on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
liuxuzxx edited a comment on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-925830058
> brokerAddrs":{0:"172.16.16.37:10911"} 地址这里也有冒号，导致解析失败的，将这里的key设置为string，就可以避免这个问题
看了下：https://www.json.org/json-en.html (不知道是不是JSON最权威的定义),从一个状态图说明了key必须是string:如图
```
从上面这个状态机图，大致明白，能解析：{},{key:value} 但是这个key看到图上标明就是string类型了
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] liuxuzxx edited a comment on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
liuxuzxx edited a comment on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-925819650
> brokerAddrs":{0:"172.16.16.37:10911"} 地址这里也有冒号，导致解析失败的，将这里的key设置为string，就可以避免这个问题
还有这个RocketMQ的获取cluster信息的JSON数据格式，不是我决定的，而是Namesrv返回的，我看到使用rocketmq团队提供的java客户端，获取这个json同样是这样子的，只不过java客户端使用的是fastjson进行解析的，但是我使用的go客户端是用encoding/json工具类解析的！所以，我无法设置这个key为string，目前倒是可以通过正则表达式替换的方式来处理!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] liuxuzxx commented on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
liuxuzxx commented on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-925816574
这个我看到是个json形式数据了，只是这个key是一个数字，不是一个字符串，按照JSON的规范定义，key应该是字符串！我实验了下，发现fastjson在序列化Map<Integer,xxxx> 这种类型的数据的时候，确实会生成这种类型的JSON，这样子是不是正常的情况(这样子可能导致JSON数据不规范，然后做不到跨语言)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] liuxuzxx commented on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
liuxuzxx commented on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-964736243
在Go当中找到了一种解决方案，总体的解决思路，就是使用JS来做个中转处理：
1. 引入如下的Go的JS引擎执行依赖
```shell
go get github.com/robertkrimen/otto
```
2. 调用JS进行序列化处理
```go
// response是获取的Cluster Information不规范的JSON结果
func BuildClusterInformation(response []byte) ClusterInformation {
vm := otto.New()
vm.Set("source", string(response))
value, err := vm.Run(`
var code = 'JSON.stringify(' + source + ')'; //拼凑字符串:JSON.stringify(source)
eval(code);//通过eval函数执行
`)
if err != nil {
log.Fatal(err.Error())
}
result, _ := value.ToString()//获取执行的结果
}
```
3. 结果比对
```json
//没有转换之前的结果：
{"brokerAddrTable":{"broker_37_master":{"brokerAddrs":{0:"172.16.16.37:10911"},"brokerName":"broker_37_master","cluster":"37_cluster"}},"clusterAddrTable":{"37_cluster":["broker_37_master"]}}
//转换之后的结果：
{"brokerAddrTable":{"broker_37_master":{"brokerAddrs":{"0":"172.16.16.37:10911"},"brokerName":"broker_37_master","cluster":"37_cluster"}},"clusterAddrTable":{"37_cluster":["broker_37_master"]}}
```
可以看到 0 这个信息已经携带上了双引号，变成了 "0"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] liuxuzxx commented on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
liuxuzxx commented on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-937082953
> @liuxuzxx 这个问题直接修改协议容易引起不兼容问题，可通过编写json解析代码规避，参考 https://github.com/apache/rocketmq-client-go/blob/master/internal/route.go#L533
好吧，暂时只能这么搞了！我实验下，多谢！具体到你说兼容问题，我觉得可以通过remotingcommand的version属性来判断决定，是否启用新的规范的json序列化！就是定义一个即将发布的版本号作为分界线，如果一个请求过来的version大于等于这个分界线版本号。那就返回key都携带双引号的json，否则，那就沿用现在的代码生成json！
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] liuxuzxx edited a comment on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
liuxuzxx edited a comment on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-925830058
> brokerAddrs":{0:"172.16.16.37:10911"} 地址这里也有冒号，导致解析失败的，将这里的key设置为string，就可以避免这个问题
看了下：https://www.json.org/json-en.html (不知道是不是JSON最权威的定义),从一个状态图说明了key必须是string:如图
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] vongosling commented on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
vongosling commented on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-926297818
感谢@liuxuzxx 对这个问题的“不一不孬“，非常棒~这个问题目前只能Go兼容下这个非标的字符了，请Go客户端几位maintainer也看下该问题，后续如何改进优化？ @ShannonDing @wenfengwang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] liuxuzxx commented on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
liuxuzxx commented on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-926309385
> 感谢@liuxuzxx 对这个问题的“不一不孬“，非常棒~这个问题目前只能Go兼容下这个非标的字符了，请Go客户端几位maintainer也看下该问题，后续如何改进优化？ @ShannonDing @wenfengwang
看到RocketMQ相关序列化组件使用的是fastjson，去fastjson下查看了下相关的issule，例如这个：https://github.com/alibaba/fastjson/issues/3821 但是好像作者并没有回应这个问题也没有关闭，不知道这个是不是fastjson的一个问题，fastjson在Map<Integer,xxx> 这种形式的序列化确实会存在不符合json规范定义的情况(目前搜索到的json规范定义要求key是string类型)，也看到讨论的issule，但是作者好像没有做回复，目前找到一种方案如下：
```java
//序列化的时候，增加SerializerFeature配置，这样子数字key就会加上双引号了
String temp = JSON.toJSONString(result, SerializerFeature.WriteNonStringKeyAsString);
```
不知道rocketmq要是这样子修改会有多大改动，只是感觉现在的这个json形式不符合规范！
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] liuxuzxx commented on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
liuxuzxx commented on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-937082953
> @liuxuzxx 这个问题直接修改协议容易引起不兼容问题，可通过编写json解析代码规避，参考 https://github.com/apache/rocketmq-client-go/blob/master/internal/route.go#L533
好吧，暂时只能这么搞了！我实验下，多谢！具体到你说兼容问题，我觉得可以通过remotingcommand的version属性来判断决定，是否启用新的规范的json序列化！就是定义一个即将发布的版本号作为分界线，如果一个请求过来的version大于等于这个分界线版本号。那就返回key都携带双引号的json，否则，那就沿用现在的代码生成json！
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] vongosling closed issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
vongosling closed issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] liuxuzxx commented on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
liuxuzxx commented on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-925817671
> brokerAddrs":{0:"172.16.16.37:10911"} 地址这里也有冒号，导致解析失败的，将这里的key设置为string，就可以避免这个问题
这个解析失败不是因为 ip地址信息含有冒号，而是因为 0 作为key，不是一个字符串形式，而是一个没有加双引号的数字形式，你如果使用：json.cn这个网站进行验证，会报错，并不是ip地址含有冒号的问题!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] liuxuzxx commented on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
liuxuzxx commented on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-925819650
> brokerAddrs":{0:"172.16.16.37:10911"} 地址这里也有冒号，导致解析失败的，将这里的key设置为string，就可以避免这个问题
还有这个RocketMQ的获取cluster信息的JSON数据格式，不是我决定的，而是Namesrv返回的，我看到使用rocketmq团队提供的java客户端，获取这个json同样是这样子的，只不过java客户端使用的是fastjson进行解析的，但是我使用的go客户端是用encoding/json工具类解析的！
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] yuz10 commented on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
yuz10 commented on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-936585515
@liuxuzxx 这个问题直接修改协议容易引起不兼容问题，可通过编写json解析代码规避，参考 https://github.com/apache/rocketmq-client-go/blob/master/internal/route.go#L533
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] liuxuzxx commented on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
liuxuzxx commented on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-925683780
why?????
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] panzhi33 commented on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
panzhi33 commented on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-925730554
brokerAddrs":{0:"172.16.16.37:10911"} 地址这里也有冒号，导致解析失败的，将这里的key设置为string，就可以避免这个问题
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Git-Yang commented on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
Git-Yang commented on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-994265719
I have fixed a related problem in go sdk PR https://github.com/apache/rocketmq-client-go/issues/733
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] liuxuzxx commented on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
liuxuzxx commented on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-925830058
> brokerAddrs":{0:"172.16.16.37:10911"} 地址这里也有冒号，导致解析失败的，将这里的key设置为string，就可以避免这个问题
看了下：https://www.json.org/json-en.html (不知道是不是JSON最权威的定义),看到里面倒并没有明确的说明key必须是字符串，不过go自己的json包处理不了这个，只能另谋他路了！
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] vongosling commented on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
vongosling commented on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-927281323
I re-opened this issue for optimization preparation. Any guys are welcome to contribute it:-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] liuxuzxx commented on issue #3369: Fetch Cluster information json data when parse in golang occure:invalid character '0' looking for beginning of object key string
Posted by GitBox <gi...@apache.org>.
liuxuzxx commented on issue #3369:
URL: https://github.com/apache/rocketmq/issues/3369#issuecomment-927259713
> 感谢@liuxuzxx 对这个问题的“不一不孬“，非常棒~这个问题目前只能Go兼容下这个非标的字符了，请Go客户端几位maintainer也看下该问题，后续如何改进优化？ @ShannonDing @wenfengwang
还有获取consumeStats信息的时候，也是JSON不规范的问题，示例JSON如下:
```json
{"consumeTps":0.0,"offsetTable":{{"brokerName":"broker_37_master","queueId":1,"topic":"MT_MQ_L4"}:{"brokerOffset":2,"consumerOffset":0,"lastTimestamp":0},{"brokerName":"broker_37_master","queueId":0,"topic":"MT_MQ_L4"}:{"brokerOffset":5,"consumerOffset":0,"lastTimestamp":0},{"brokerName":"broker_37_master","queueId":3,"topic":"MT_MQ_L4"}:{"brokerOffset":2,"consumerOffset":0,"lastTimestamp":0},{"brokerName":"broker_37_master","queueId":2,"topic":"MT_MQ_L4"}:{"brokerOffset":2,"consumerOffset":0,"lastTimestamp":0}}}
```
这次和上一个问题的key没有携带双引号还有很大区别，这个是直接一个对象的JSON串当做一个key了
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org