You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2026/08/27 17:35:19 UTC
[PR] perf(parquet): compile `ValueStatistics` `Display`/`Debug` formatting once, not per type [arrow-rs]
alamb opened a new pull request, #10892:
URL: https://github.com/apache/arrow-rs/pull/10892
# Which issue does this PR close?
- Part of the binary-size / LLVM IR reduction effort started in #10889.
# Rationale for this change
`Display for ValueStatistics<T>` is monomorphized for all 8 `ParquetValueType`s, and each copy is ~2,000 lines of LLVM IR (~16,000 lines total in the `parquet` lib) even though only the `min`/`max` writes actually depend on `T`. `Debug` has the same shape. Statistics formatting is not performance-sensitive, so this is a pure code-size / compile-time win.
# What changes are included in this PR?
Introduce a private `ErasedValueStatistics` struct that borrows the statistics with `min`/`max` behind trait objects. The generic `Display`/`Debug` impls shrink to constructing it and delegating; the (unchanged) formatting bodies move to non-generic methods on it, compiled once. Output is byte-for-byte identical.
Measured with `cargo llvm-lines --release -p parquet --features arrow --lib` (cargo-llvm-lines 0.4.41, aarch64-apple-darwin):
| | before | after |
|---|---:|---:|
| `ValueStatistics` formatting IR | 16,168 lines (8 × ~2,020) | ~3,100 lines (one shared body + 8 thin shims) **(−81%)** |
# Are these changes tested?
Covered by existing tests; the format strings are unchanged.
# Are there any user-facing changes?
No. `ErasedValueStatistics` is private and the formatted output is identical.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(parquet): compile `ValueStatistics` `Display`/`Debug` formatting once, not per type [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed pull request #10892: perf(parquet): compile `ValueStatistics` `Display`/`Debug` formatting once, not per type
URL: https://github.com/apache/arrow-rs/pull/10892
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(parquet): compile `ValueStatistics` `Display`/`Debug` formatting once, not per type [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #10892:
URL: https://github.com/apache/arrow-rs/pull/10892#issuecomment-5444219090
Turns out LLVM already dedupes this code and the binary savings is negligible. so closing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org