You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "GoGoWen (via GitHub)" <gi...@apache.org> on 2023/06/30 09:59:05 UTC
[GitHub] [doris] GoGoWen opened a new pull request, #21390: [Fix](JSON LOAD)fix json load issue when string with RFC 4627
GoGoWen opened a new pull request, #21390:
URL: https://github.com/apache/doris/pull/21390
## Proposed changes
Issue Number: close #21389
from rapidjson:
Query String
In addition to GetString(), the Value class also contains GetStringLength(). Here explains why:
According to RFC 4627, JSON strings can contain Unicode character U+0000, which must be escaped as "\u0000". The problem is that, C/C++ often uses null-terminated string, which treats \0 as the terminator symbol.
To conform with RFC 4627, RapidJSON supports string containing U+0000 character. If you need to handle this, you can use GetStringLength() to obtain the correct string length.
For example, after parsing the following JSON to Document d:
{ "s" : "a\u0000b" }
The correct length of the string "a\u0000b" is 3, as returned by GetStringLength(). But strlen() returns 1.
GetStringLength() can also improve performance, as user may often need to call strlen() for allocating buffer.
Besides, std::string also support a constructor:
string(const char* s, size_t count);
which accepts the length of string as parameter. This constructor supports storing null character within the string, and should also provide better performance.
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] eldenmoon commented on pull request #21390: [Fix](JSON LOAD)fix json load issue when string conform with RFC 4627
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon commented on PR #21390:
URL: https://github.com/apache/doris/pull/21390#issuecomment-1614644464
could you add regreesion test?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21390: [Fix](JSON LOAD)fix json load issue when string conform with RFC 4627
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21390:
URL: https://github.com/apache/doris/pull/21390#issuecomment-1614435146
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] GoGoWen commented on pull request #21390: [Fix](JSON LOAD)fix json load issue when string conform with RFC 4627
Posted by "GoGoWen (via GitHub)" <gi...@apache.org>.
GoGoWen commented on PR #21390:
URL: https://github.com/apache/doris/pull/21390#issuecomment-1620983612
> could you add regreesion test?
the regression is hard to add as need to set be config enable_simdjson_reader=false, seems master code is use simdjson by default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #21390: [Fix](JSON LOAD)fix json load issue when string conform with RFC 4627
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #21390:
URL: https://github.com/apache/doris/pull/21390
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] GoGoWen commented on pull request #21390: [Fix](JSON LOAD)fix json load issue when string conform with RFC 4627
Posted by "GoGoWen (via GitHub)" <gi...@apache.org>.
GoGoWen commented on PR #21390:
URL: https://github.com/apache/doris/pull/21390#issuecomment-1614454441
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21390: [Fix](JSON LOAD)fix json load issue when string conform with RFC 4627
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21390:
URL: https://github.com/apache/doris/pull/21390#issuecomment-1621435487
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21390: [Fix](JSON LOAD)fix json load issue when string conform with RFC 4627
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21390:
URL: https://github.com/apache/doris/pull/21390#issuecomment-1621435554
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org