You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "wgtmac (via GitHub)" <gi...@apache.org> on 2025/01/07 05:27:39 UTC

[PR] AVRO-4107: [C++] Remove boost::algorithm [avro]

wgtmac opened a new pull request, #3284:
URL: https://github.com/apache/avro/pull/3284

   ## What is the purpose of the change
   
   Remove boost::algorithm
   
   ## Verifying this change
   
   This change is a trivial rework / code cleanup without any test coverage.
   
   ## Documentation
   
   - Does this pull request introduce a new feature? no
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@avro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] AVRO-4107: [C++] Remove boost::algorithm [avro]

Posted by "KalleOlaviNiemitalo (via GitHub)" <gi...@apache.org>.
KalleOlaviNiemitalo commented on code in PR #3284:
URL: https://github.com/apache/avro/pull/3284#discussion_r1905592160


##########
lang/c++/impl/avrogencpp.cc:
##########
@@ -859,12 +869,19 @@ static string readGuard(const string &filename) {
     string buf;
     string candidate;
     while (std::getline(ifs, buf)) {
-        boost::algorithm::trim(buf);
+        if (!buf.empty()) {
+            size_t start = 0, end = buf.length();
+            while (start < end && std::isspace(buf[start])) start++;
+            while (start < end && std::isspace(buf[end - 1])) end--;

Review Comment:
   If `buf[start]` or `buf[end - 1]` is a negative `char` other than `EOF`, then passing it to `std::isspace(int ch)` causes undefined behavior.  To avoid the risk, this should cast the `char` values to `unsigned char`.
   
   (In principle, this might also be expected to recognize and trim multibyte whitespace characters as defined by the current locale.  But it seems unlikely that such characters would actually be used in the preprocessor directives for which this function searches.  And `boost::algorithm::trim` doesn't seem to recognize them either, as it checks each `char` individually.)



##########
lang/c++/impl/avrogencpp.cc:
##########
@@ -859,12 +869,19 @@ static string readGuard(const string &filename) {
     string buf;
     string candidate;
     while (std::getline(ifs, buf)) {
-        boost::algorithm::trim(buf);
+        if (!buf.empty()) {
+            size_t start = 0, end = buf.length();
+            while (start < end && std::isspace(buf[start])) start++;
+            while (start < end && std::isspace(buf[end - 1])) end--;
+            if (start > 0 || end < buf.length()) {
+                buf = buf.substr(start, end - start);
+            }
+        }
         if (candidate.empty()) {
-            if (boost::algorithm::starts_with(buf, "#ifndef ")) {
+            if (buf.substr(0, 8) == "#ifndef ") {

Review Comment:
   A remark rather than a recommendation: `buf.compare(0, 8, "#ifndef ") == 0` might be faster here, because of less copying.  But also a bit harder to understand and the performance difference might not be significant.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] AVRO-4107: [C++] Remove boost::algorithm [avro]

Posted by "thiru-mg (via GitHub)" <gi...@apache.org>.
thiru-mg commented on PR #3284:
URL: https://github.com/apache/avro/pull/3284#issuecomment-2583009060

   @wgtmac @KalleOlaviNiemitalo 
   
   I think the version that @wgtmac implemented (which uses locale) is the closest to what boost implemented. Boost's trim also uses the current locale for determining if something is a space.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] AVRO-4107: [C++] Remove boost::algorithm [avro]

Posted by "KalleOlaviNiemitalo (via GitHub)" <gi...@apache.org>.
KalleOlaviNiemitalo commented on code in PR #3284:
URL: https://github.com/apache/avro/pull/3284#discussion_r1905024236


##########
lang/c++/impl/Compiler.cc:
##########
@@ -136,7 +136,11 @@ int64_t getLongField(const Entity &e, const Object &m,
 // Unescape double quotes (") for de-serialization.  This method complements the
 // method NodeImpl::escape() which is used for serialization.
 static void unescape(string &s) {
-    boost::replace_all(s, "\\\"", "\"");
+    std::string::size_type pos = 0;
+    while ((pos = s.find("\\\"", pos)) != std::string::npos) {
+        s.replace(pos, 2, "\"");
+        pos += 2;

Review Comment:
   This is wrong; `pos` should be incremented by 1, not 2.  Because the replacement string `"\""` has only one character.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] AVRO-4107: [C++] Remove boost::algorithm [avro]

Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on code in PR #3284:
URL: https://github.com/apache/avro/pull/3284#discussion_r1909836549


##########
lang/c++/impl/avrogencpp.cc:
##########
@@ -859,12 +869,19 @@ static string readGuard(const string &filename) {
     string buf;
     string candidate;
     while (std::getline(ifs, buf)) {
-        boost::algorithm::trim(buf);
+        if (!buf.empty()) {
+            size_t start = 0, end = buf.length();
+            while (start < end && std::isspace(buf[start])) start++;
+            while (start < end && std::isspace(buf[end - 1])) end--;

Review Comment:
   I'm not sure if it is better to call `std::isspace(buf[start], std::locale::classic())` instead of `std::isspace(static_cast<unsigned char>(buf[start]))`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] AVRO-4107: [C++] Remove boost::algorithm [avro]

Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on code in PR #3284:
URL: https://github.com/apache/avro/pull/3284#discussion_r1909836049


##########
lang/c++/impl/Compiler.cc:
##########
@@ -136,7 +136,11 @@ int64_t getLongField(const Entity &e, const Object &m,
 // Unescape double quotes (") for de-serialization.  This method complements the
 // method NodeImpl::escape() which is used for serialization.

Review Comment:
   Make sense. If this is really an issue, we can fix it in a separate PR.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] AVRO-4107: [C++] Remove boost::algorithm [avro]

Posted by "KalleOlaviNiemitalo (via GitHub)" <gi...@apache.org>.
KalleOlaviNiemitalo commented on code in PR #3284:
URL: https://github.com/apache/avro/pull/3284#discussion_r1904943793


##########
lang/c++/impl/Compiler.cc:
##########
@@ -136,7 +136,11 @@ int64_t getLongField(const Entity &e, const Object &m,
 // Unescape double quotes (") for de-serialization.  This method complements the
 // method NodeImpl::escape() which is used for serialization.
 static void unescape(string &s) {
-    boost::replace_all(s, "\\\"", "\"");
+    std::string::size_type pos = 0;
+    while ((pos = s.find("\\\"", pos)) != std::string::npos) {
+        s.replace(pos, 2, "\"");
+        pos += 2;
+    }

Review Comment:
   This loop looks like it has O(n^2) time complexity.  Does boost::replace_all have the same or only O(n)?
   
   OTOH, this is in the schema compiler and not in data processing so perhaps it doesn't matter.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] AVRO-4107: [C++] Remove boost::algorithm [avro]

Posted by "KalleOlaviNiemitalo (via GitHub)" <gi...@apache.org>.
KalleOlaviNiemitalo commented on code in PR #3284:
URL: https://github.com/apache/avro/pull/3284#discussion_r1905056392


##########
lang/c++/impl/Compiler.cc:
##########
@@ -136,7 +136,11 @@ int64_t getLongField(const Entity &e, const Object &m,
 // Unescape double quotes (") for de-serialization.  This method complements the
 // method NodeImpl::escape() which is used for serialization.

Review Comment:
   A side note.  The NodeImpl::escape function referenced from here calls NodeImpl::intToHex:
   
   <https://github.com/apache/avro/blob/dd01f97869e374d6427bec999afa089f760791ab/lang/c%2B%2B/impl/NodeImpl.cc#L59-L65>
   
   <https://github.com/apache/avro/blob/dd01f97869e374d6427bec999afa089f760791ab/lang/c%2B%2B/include/avro/NodeImpl.hh#L543-L550>
   
   In NodeImpl::intToHex, the `std::setw(sizeof(T))` call looks wrong for two reasons:
   
   * If bytes are 8-bit, then each byte makes two hexadecimal digits, not 1.
   * According to <https://datatracker.ietf.org/doc/html/rfc7159#section-7>, `\u` in JSON must always be followed by four hexadecimal digits; this does not depend on which C++ type was used.
   
   But I suppose it works in practice because `T` is either `int` or `unsigned int` and `sizeof(T)` is 4 on usual architectures and the actual value of `T i` is cast from `char` or `uint8_t` so it is 0xFF or less.  Although this could break if `char` is a signed type and `std::iscntrl` returns true for a negative value.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] AVRO-4107: [C++] Remove boost::algorithm [avro]

Posted by "thiru-mg (via GitHub)" <gi...@apache.org>.
thiru-mg merged PR #3284:
URL: https://github.com/apache/avro/pull/3284


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@avro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] AVRO-4107: [C++] Remove boost::algorithm [avro]

Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on PR #3284:
URL: https://github.com/apache/avro/pull/3284#issuecomment-2581784540

   @thiru-mg @martin-g @Fokko Could you help review this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] AVRO-4107: [C++] Remove boost::algorithm [avro]

Posted by "KalleOlaviNiemitalo (via GitHub)" <gi...@apache.org>.
KalleOlaviNiemitalo commented on code in PR #3284:
URL: https://github.com/apache/avro/pull/3284#discussion_r1904945479


##########
lang/c++/impl/Compiler.cc:
##########
@@ -136,7 +136,11 @@ int64_t getLongField(const Entity &e, const Object &m,
 // Unescape double quotes (") for de-serialization.  This method complements the
 // method NodeImpl::escape() which is used for serialization.
 static void unescape(string &s) {
-    boost::replace_all(s, "\\\"", "\"");
+    std::string::size_type pos = 0;
+    while ((pos = s.find("\\\"", pos)) != std::string::npos) {
+        s.replace(pos, 2, "\"");
+        pos += 2;
+    }

Review Comment:
   O(n^2) because each iteration removes one character from the string and has to move all the following characters.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] AVRO-4107: [C++] Remove boost::algorithm [avro]

Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on code in PR #3284:
URL: https://github.com/apache/avro/pull/3284#discussion_r1905535502


##########
lang/c++/impl/Compiler.cc:
##########
@@ -136,7 +136,11 @@ int64_t getLongField(const Entity &e, const Object &m,
 // Unescape double quotes (") for de-serialization.  This method complements the
 // method NodeImpl::escape() which is used for serialization.
 static void unescape(string &s) {
-    boost::replace_all(s, "\\\"", "\"");
+    std::string::size_type pos = 0;
+    while ((pos = s.find("\\\"", pos)) != std::string::npos) {
+        s.replace(pos, 2, "\"");
+        pos += 2;
+    }

Review Comment:
   I have changed it to O(n). Please take a look again. Thanks!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] AVRO-4107: [C++] Remove boost::algorithm [avro]

Posted by "KalleOlaviNiemitalo (via GitHub)" <gi...@apache.org>.
KalleOlaviNiemitalo commented on code in PR #3284:
URL: https://github.com/apache/avro/pull/3284#discussion_r1905577124


##########
lang/c++/impl/Compiler.cc:
##########
@@ -136,7 +136,11 @@ int64_t getLongField(const Entity &e, const Object &m,
 // Unescape double quotes (") for de-serialization.  This method complements the
 // method NodeImpl::escape() which is used for serialization.
 static void unescape(string &s) {
-    boost::replace_all(s, "\\\"", "\"");
+    std::string::size_type pos = 0;
+    while ((pos = s.find("\\\"", pos)) != std::string::npos) {
+        s.replace(pos, 2, "\"");
+        pos += 2;
+    }

Review Comment:
   Yes, 35a9b5a1a3a0f0a7589c2ef63f71e899ce1c01a9 looks all right in that respect.
   
   It is somewhat strange that the `unescape` function translates `\"` to `"` but does nothing to `\\`.  However, this pull request does not change that behavior, and it is not related to whether Boost is used.  So if it needs fixing, I guess it's better to do that under a separate issue.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org