Re: [PR] fix(parquet): support mask filtering across skipped pages [arrow-rs]

["hhhizzz (via GitHub)" <gi...@apache.org>]

hhhizzz commented on PR #10288:
URL: https://github.com/apache/arrow-rs/pull/10288#issuecomment-4998265375

   @alamb 👋 A brief update since your last note: I simplified the implementation
   and added the remaining regression coverage for batch shape and nested columns
   with different page boundaries.
   
   I also realized that, in earlier iterations, I let the scope grow by pursuing
   too many detailed performance improvements at once. AI-assisted tools make it
   easy to explore and implement related ideas quickly, but they do not reduce the
   reviewer's cognitive load. That scope was therefore asking too much from
   reviewers.
   
   I have now narrowed this PR to one behavioral goal: making Mask execution
   correct across sparse pages without falling back to selectors. Future
   optimizations can be evaluated and reviewed separately in focused follow-up
   PRs. This is also how I plan to scope my contributions going forward: establish
   the behavioral change first, then optimize incrementally with separate evidence.
   
   The PR has 604 additions and 168 deletions overall, but when `#[cfg(test)]`
   code is counted as tests, **_the core runtime change is +244/-116 across four
   files, or a net increase of 128 lines._** Tests are +360/-52 and account for most
   of the net growth. I also revised the description around the `LoadedRowRanges`
   invariant and how safe decode chunks are coalesced into one logical batch.
   All current CI checks are green ✅
   
   Whenever your schedule allows, I’d really appreciate your review of the
   approach 🙏 I’d like to keep building context in Parquet predicate pushdown
   and gradually help maintain this area, including reviewing related issues and
   PRs and taking follow-up work. Please feel free to tag me wherever I can be
   useful. Thank you! 😊


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org