[I] Unify Parquet `PageIndexPolicy` with `ColumnChunkMask` [arrow-rs]

["etseidl (via GitHub)" <gi...@apache.org>]

etseidl opened a new issue, #11395:
URL: https://github.com/apache/arrow-rs/issues/11395

   ### Is your feature request related to a problem or challenge?
   
   This is a followup to a [comment](https://github.com/apache/arrow-rs/pull/11157#discussion_r4144630473) in #11157.
   
   >The interaction between PageIndexPolicy and the ColumnChunkMask is complicated, especially when there is an existing PageIndex in ParquetMetadata
   >
   >One potential API we could consider is a new variant in `PageIndexPolicy::Mask` to avoid having to thread through another set of variables. 
   >
   >```rust
   >pub enum PageIndexPolicy {
   >    /// Do not read the page index.
   >    #[default]
   >    Skip,
   >...
   >    /// Ensure the specified indexes exist, and error if not
   >    Required(ColumnChunkMask),
   >    /// Load the specified indexes if they exist
   >    Optional(ColumnChunkMask),
   >}
   >```
   >
   >However, that would be an API change as `PageIndexPolicy` isn't marked as `non_exhastive` so we would have to either 
   >1. Wait for the next major release
   >2. Go with this API, and then unify PageIndexPolicy, and deprecate `with_column_index_mask` et al methods  in next major version 🤔 
   
   ### Describe the solution you'd like
   
   #11157 will ship with the policy and mask specified separately, so this represents option 2 above. The goal is to modify `PageIndexPolicy` either as above, or potentially via new variants, to provide the chunk masks along with the policy. This will be a breaking API change so it will have to wait for 61.0.0.
   
   ### Describe alternatives you've considered
   
   Keep the existing API, but that is pretty cluttered already. It's best to have a single mechanism to specify how to build the page indexes.
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org