You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@servicecomb.apache.org by GitBox <gi...@apache.org> on 2022/07/15 08:44:20 UTC
[GitHub] [servicecomb-java-chassis] sleep4u opened a new issue, #3198: RPC调用时body中有emoji等UTF-8编码时buffer会设置异常
sleep4u opened a new issue, #3198:
URL: https://github.com/apache/servicecomb-java-chassis/issues/3198
body中存在emoji表情时
BodyProcessorCreator里的createBodyBuffer方法返回的buffer设置的有问题
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@servicecomb.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] sleep4u commented on issue #3198: RPC调用时body中有emoji等UTF-8编码时buffer会设置异常
Posted by GitBox <gi...@apache.org>.
sleep4u commented on issue #3198:
URL: https://github.com/apache/servicecomb-java-chassis/issues/3198#issuecomment-1185339900
补充一个demo
```java
@Test
public void test() throws Exception {
TextMessage message = new TextMessage("📣");
System.out.println(Arrays.toString("📣".getBytes(StandardCharsets.UTF_8)));
System.out.println(Arrays.toString("\uD83D\uDCE3".getBytes(StandardCharsets.UTF_8)));
// 转义后每个字符作为String
System.out.println(Arrays.toString("\\uD83D\\uDCE3".getBytes(StandardCharsets.UTF_8)));
Buffer requestBody;
try (BufferOutputStream output = new BufferOutputStream()) {
RestObjectMapperFactory.getConsumerWriterMapper().writeValue(output, message);
requestBody = output.getBuffer();
}
System.out.println(Arrays.toString(requestBody.getBytes()));
}
```
结果是
[-16, -97, -109, -93]
[-16, -97, -109, -93]
[92, 117, 68, 56, 51, 68, 92, 117, 68, 67, 69, 51]
[123, 34, 116, 34, 58, 34, **92, 117, 68, 56, 51, 68, 92, 117, 68, 67, 69, 51**, 34, 125]
可以看到buffer里的emoji变成了错误的内容，当成了非unicode来处理
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@servicecomb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] sleep4u commented on issue #3198: RPC调用时body中有emoji等unicode时buffer转换会无法识别unicode
Posted by GitBox <gi...@apache.org>.
sleep4u commented on issue #3198:
URL: https://github.com/apache/servicecomb-java-chassis/issues/3198#issuecomment-1188567520
追到了这里UTF8JsonGenerator#_outputMultiByteChar
CSE现在这种encode decode流程会丢失unicode
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@servicecomb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] sleep4u closed issue #3198: RPC调用时body中有emoji等unicode时buffer转换会无法识别unicode
Posted by GitBox <gi...@apache.org>.
sleep4u closed issue #3198: RPC调用时body中有emoji等unicode时buffer转换会无法识别unicode
URL: https://github.com/apache/servicecomb-java-chassis/issues/3198
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@servicecomb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] lbc97 commented on issue #3198: RPC调用时body中有emoji等unicode时buffer转换会无法识别unicode
Posted by GitBox <gi...@apache.org>.
lbc97 commented on issue #3198:
URL: https://github.com/apache/servicecomb-java-chassis/issues/3198#issuecomment-1192152228
> 补充一个demo
>
> ```java
> @Test
> public void test() throws Exception {
>
> TextMessage message = new TextMessage("📣");
> System.out.println(Arrays.toString("📣".getBytes(StandardCharsets.UTF_8)));
> System.out.println(Arrays.toString("\uD83D\uDCE3".getBytes(StandardCharsets.UTF_8)));
> // 转义后每个字符作为String
> System.out.println(Arrays.toString("\\uD83D\\uDCE3".getBytes(StandardCharsets.UTF_8)));
> Buffer requestBody;
> try (BufferOutputStream output = new BufferOutputStream()) {
> RestObjectMapperFactory.getConsumerWriterMapper().writeValue(output, message);
> requestBody = output.getBuffer();
> }
> System.out.println(Arrays.toString(requestBody.getBytes()));
> }
> ```
>
> 结果是 [-16, -97, -109, -93] [-16, -97, -109, -93] [92, 117, 68, 56, 51, 68, 92, 117, 68, 67, 69, 51] [123, 34, 116, 34, 58, 34, **92, 117, 68, 56, 51, 68, 92, 117, 68, 67, 69, 51**, 34, 125]
>
> 可以看到buffer里的emoji变成了错误的内容，当成了非unicode来处理
你那个TextMessage是啥啊
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@servicecomb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] sleep4u commented on issue #3198: RPC调用时body中有emoji等unicode时buffer转换会无法识别unicode
Posted by GitBox <gi...@apache.org>.
sleep4u commented on issue #3198:
URL: https://github.com/apache/servicecomb-java-chassis/issues/3198#issuecomment-1197711683
> > 补充一个demo
> > ```java
> > @Test
> > public void test() throws Exception {
> >
> > TextMessage message = new TextMessage("📣");
> > System.out.println(Arrays.toString("📣".getBytes(StandardCharsets.UTF_8)));
> > System.out.println(Arrays.toString("\uD83D\uDCE3".getBytes(StandardCharsets.UTF_8)));
> > // 转义后每个字符作为String
> > System.out.println(Arrays.toString("\\uD83D\\uDCE3".getBytes(StandardCharsets.UTF_8)));
> > Buffer requestBody;
> > try (BufferOutputStream output = new BufferOutputStream()) {
> > RestObjectMapperFactory.getConsumerWriterMapper().writeValue(output, message);
> > requestBody = output.getBuffer();
> > }
> > System.out.println(Arrays.toString(requestBody.getBytes()));
> > }
> > ```
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > 结果是 [-16, -97, -109, -93] [-16, -97, -109, -93] [92, 117, 68, 56, 51, 68, 92, 117, 68, 67, 69, 51] [123, 34, 116, 34, 58, 34, **92, 117, 68, 56, 51, 68, 92, 117, 68, 67, 69, 51**, 34, 125]
> > 可以看到buffer里的emoji变成了错误的内容，当成了非unicode来处理
>
> 你那个TextMessage是啥啊
就是个model
@Data
@NoArgsConstructor
public class TextMessage {
private String t;
public TextMessage(String text) {
t = text;
}
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@servicecomb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] sleep4u commented on issue #3198: RPC调用时body中有emoji等unicode时buffer转换会无法识别unicode
Posted by GitBox <gi...@apache.org>.
sleep4u commented on issue #3198:
URL: https://github.com/apache/servicecomb-java-chassis/issues/3198#issuecomment-1197720835
这个问题已经明确。
jackson对部分unicode的处理和String对unicode的处理不一样，不能说哪种不对
CSE之间有配套的编解码。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@servicecomb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org