You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "wenn-id (via GitHub)" <gi...@apache.org> on 2026/07/29 02:02:59 UTC
[PR] Fix Arrow narrow integer bloom filter checks [arrow-rs]
wenn-id opened a new pull request, #10467:
URL: https://github.com/apache/arrow-rs/pull/10467
Closes #5550.
# Rationale for this change
Arrow `Int8`, `Int16`, `UInt8`, and `UInt16` values are widened to Parquet `INT32` when written. Checking their original narrow byte representation against the serialized bloom filter therefore produces false negatives.
# What changes are included in this PR?
- Add an Arrow-aware array kernel that widens narrow integer arrays to their Parquet physical representation before checking `Sbbf`.
- Share the narrow-integer coercion helper with `ArrowWriter`, keeping writer and checker semantics aligned.
- Preserve Arrow nulls in the returned `BooleanArray` and reject unsupported Arrow types.
- Add round-trip tests using bloom filters written by `ArrowWriter`, serialized, reopened, and checked for present, absent, boundary, and null values.
# Are there any user-facing changes?
Adds `parquet::arrow::bloom_filter::check_bloom_filter` for `Int8`, `Int16`, `UInt8`, and `UInt16` arrays. Other Arrow-to-Parquet coercions remain unsupported and return an error.
# Testing
- `cargo test -p parquet --lib --features arrow` (1,237 passed)
- `cargo test -p parquet --lib bloom_filter --features arrow -- --nocapture` (38 passed)
- `cargo clippy -p parquet --lib --all-features -- -D warnings`
- `RUSTDOCFLAGS="-D warnings" cargo doc -p parquet --all-features --no-deps`
- `cargo +stable fmt --all -- --check`
- `git diff --check`
# AI usage
Hermes Agent was used to help analyze the existing code and draft the implementation and tests. I reviewed the complete diff, added independent negative-membership coverage, initialized the official test-data submodules, ran the full Parquet library test suite and quality gates above, and can maintain and debug the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Arrow narrow integer bloom filter checks [arrow-rs]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10467:
URL: https://github.com/apache/arrow-rs/pull/10467#issuecomment-5975759826
Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Arrow narrow integer bloom filter checks [arrow-rs]
Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on PR #10467:
URL: https://github.com/apache/arrow-rs/pull/10467#issuecomment-5111959760
i find it very curious that within 10 minutes of me updating the original issue it was picked up, then 12 minutes after a full PR is prepared and submitted 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Arrow narrow integer bloom filter checks [arrow-rs]
Posted by "wenn-id (via GitHub)" <gi...@apache.org>.
wenn-id closed pull request #10467: Fix Arrow narrow integer bloom filter checks
URL: https://github.com/apache/arrow-rs/pull/10467
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org