You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "pedrobtz (via GitHub)" <gi...@apache.org> on 2026/10/21 21:54:00 UTC
[PR] fix(ipc): Zero the padding of footer Blocks [arrow-nanoarrow]
pedrobtz opened a new pull request, #986:
URL: https://github.com/apache/arrow-nanoarrow/pull/986
`ArrowIpcEncoderEncodeFooter()` builds each `Block` of the footer on the stack with an initializer, then copies it into the flatbuffer vector. `Block` has four bytes of padding after `metaDataLength`, and an initializer doesn't set padding. So the footer carried whatever bytes were on the stack. That makes files non-reproducible, and the stack bytes end up in the file. With Apple clang 17 at `-O0`, filling the stack with `0xAB` before the call gives padding `abababab`.
The vector is now zeroed after `extend`, and each element is set with the generated `Block_assign()`. Both the record batch and the dictionary block loops are changed.
The new `NanoarrowIpcFooterEncodingIsDeterministic` test encodes the same footer twice, after filling the stack with `0x00` and then `0xFF`, and requires identical bytes. In a plain Debug build it fails without the fix and passes with it. Under ASan it passes either way, because ASan moves stack frames.
Found through a byte-for-byte determinism test in the [zuarrow](https://github.com/pedrobtz/zuarrow) R package, which vendors nanoarrow.
`ctest` passes in a plain Debug build and with ASan and UBSan (410 tests each). The changed files are formatted with the pinned clang-format 16.0.6.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(ipc): Zero the padding of footer Blocks [arrow-nanoarrow]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #986:
URL: https://github.com/apache/arrow-nanoarrow/pull/986#issuecomment-6407772184
## [Codecov](https://app.codecov.io/gh/apache/arrow-nanoarrow/pull/986?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 78.59%. Comparing base ([`55df060`](https://app.codecov.io/gh/apache/arrow-nanoarrow/commit/55df06001d6c13294642fa3b741ac4e316553b3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`0934365`](https://app.codecov.io/gh/apache/arrow-nanoarrow/commit/09343650e5a7ff1153c1cd65a5f3a3d67226c296?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 4 commits behind head on main.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #986 +/- ##
==========================================
- Coverage 78.59% 78.59% -0.01%
==========================================
Files 106 106
Lines 17153 17149 -4
Branches 2043 2043
==========================================
- Hits 13482 13478 -4
Misses 2422 2422
Partials 1249 1249
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/arrow-nanoarrow/pull/986?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(ipc): Zero the padding of footer Blocks [arrow-nanoarrow]
Posted by "pedrobtz (via GitHub) via github" <gi...@arrow.apache.org>.
pedrobtz commented on PR #986:
URL: https://github.com/apache/arrow-nanoarrow/pull/986#issuecomment-6797951452
Thanks! I've dropped the test, so the PR is now just the fix in `encoder.c`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(ipc): Zero the padding of footer Blocks [arrow-nanoarrow]
Posted by "paleolimbot (via GitHub) via github" <gi...@arrow.apache.org>.
paleolimbot commented on code in PR #986:
URL: https://github.com/apache/arrow-nanoarrow/pull/986#discussion_r4534300985
##########
src/nanoarrow/ipc/encoder_test.cc:
##########
@@ -108,6 +108,55 @@ TEST(NanoarrowIpcTest, NanoarrowIpcFooterEncoding) {
EXPECT_GT(footer_buffer->size_bytes, raw_schema_buffer->size_bytes);
}
+// Fill a stretch of the stack so that uninitialized stack bytes used by a
+// subsequent call are not zero by chance
+#if defined(_MSC_VER)
+__declspec(noinline)
+#else
Review Comment:
I appreciate the thoroughness here, but this test still passes in most cases even without the fix. I think this PR is stronger without the test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(ipc): Zero the padding of footer Blocks [arrow-nanoarrow]
Posted by "paleolimbot (via GitHub) via github" <gi...@arrow.apache.org>.
paleolimbot merged PR #986:
URL: https://github.com/apache/arrow-nanoarrow/pull/986
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org