[GitHub] [parquet-mr] parthchandra commented on pull request #1139: PARQUET-2171: Support Hadoop vectored IO

["parthchandra (via GitHub)" <gi...@apache.org>]

parthchandra commented on PR #1139:
URL: https://github.com/apache/parquet-mr/pull/1139#issuecomment-1739701945

   > @danielcweeks that's a good point about pluggability.
   > I don't know if that would be useful for iceberg https://github.com/apache/hadoop-api-shim
   
   Iceberg can use the base Parquet File reader out of the box so should be able to use vector IO as it is. 
   
   > getting iceberg to pass down which stripes it wants to read is critical for this to work best with s3, abfs and gcs. how are you reading the files at present?
   
   However if the S3FileIO feature is enabled, Iceberg provides its own InputStream and InputFile implementation that use AWS SDK V2. Maybe an option to provide your own input stream to vector io might work.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org