You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "fallintoplace (via GitHub)" <gi...@apache.org> on 2026/10/20 11:26:19 UTC
[PR] perf(util): encode Base64 in three-byte blocks [iceberg-cpp]
fallintoplace opened a new pull request, #1045:
URL: https://github.com/apache/iceberg-cpp/pull/1045
**What**
- Encode Base64 input in 3-byte blocks.
**Why**
- Reduce per-byte work during encoding.
**Implementation**
- Pre-size the output and write encoded blocks directly.
- Keep standard and URL-safe output.
**Benchmark**
- Standalone GCC 14 `-O3`: 1 KiB 2.427 → 0.811 µs (3.0x); 16 KiB 37.55 → 13.49 µs (2.8x).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] perf(util): encode Base64 in three-byte blocks [iceberg-cpp]
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac merged PR #1045:
URL: https://github.com/apache/iceberg-cpp/pull/1045
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] perf(util): encode Base64 in three-byte blocks [iceberg-cpp]
Posted by "zhjwpku (via GitHub)" <gi...@apache.org>.
zhjwpku commented on code in PR #1045:
URL: https://github.com/apache/iceberg-cpp/pull/1045#discussion_r4324703689
##########
src/iceberg/util/base64.cc:
##########
@@ -30,28 +30,42 @@ namespace {
// When `pad` is true, the output is padded with '=' to a multiple of 4 characters.
std::string Base64EncodeWithAlphabet(std::string_view input, std::string_view alphabet,
bool pad) {
- std::string output;
- output.reserve((input.size() + 2) / 3 * 4);
+ const size_t full_blocks = input.size() / 3;
+ const size_t remainder = input.size() % 3;
+ const size_t output_size =
+ full_blocks * 4 + (remainder == 0 ? 0 : (pad ? 4 : remainder + 1));
+ std::string output(output_size, '\0');
+
+ const auto* bytes = reinterpret_cast<const uint8_t*>(input.data());
+ size_t out = 0;
+ for (size_t i = 0; i < full_blocks * 3; i += 3) {
+ const uint32_t block = (static_cast<uint32_t>(bytes[i]) << 16) |
+ (static_cast<uint32_t>(bytes[i + 1]) << 8) |
+ static_cast<uint32_t>(bytes[i + 2]);
+ output[out++] = alphabet[(block >> 18) & 0x3F];
+ output[out++] = alphabet[(block >> 12) & 0x3F];
+ output[out++] = alphabet[(block >> 6) & 0x3F];
+ output[out++] = alphabet[block & 0x3F];
+ }
- uint32_t buffer = 0;
- int bits_collected = 0;
- for (unsigned char byte : input) {
- buffer = (buffer << 8) | byte;
- bits_collected += 8;
- while (bits_collected >= 6) {
- bits_collected -= 6;
- output.push_back(alphabet[(buffer >> bits_collected) & 0x3F]);
+ if (remainder > 0) {
+ const size_t i = full_blocks * 3;
+ uint32_t block = static_cast<uint32_t>(bytes[i]) << 16;
+ if (remainder == 2) {
+ block |= static_cast<uint32_t>(bytes[i + 1]) << 8;
}
- }
- if (bits_collected > 0) {
- // Pad the remaining bits on the right to form the final 6-bit group.
- output.push_back(alphabet[(buffer << (6 - bits_collected)) & 0x3F]);
- }
- if (pad) {
- while (output.size() % 4 != 0) {
- output.push_back('=');
+ output[out++] = alphabet[(block >> 18) & 0x3F];
+ output[out++] = alphabet[(block >> 12) & 0x3F];
+ if (remainder == 2) {
+ output[out++] = alphabet[(block >> 6) & 0x3F];
+ }
+ if (pad) {
Review Comment:
nit: seems we can delete this condition wrapper since the output_size is exact.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org