You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Zeeyi13 (via GitHub)" <gi...@apache.org> on 2024/08/30 01:47:47 UTC

[I] [Go] Need help on reading parquet from S3 [arrow-go]

Zeeyi13 opened a new issue, #37:
URL: https://github.com/apache/arrow-go/issues/37

   Hi team,
   
   I would like to read a parquet file from S3 with high performance. Is there any hit or an example for me to start with? I have some ideas , but not sure which one is recommended or any better solutions?
   
   One approach is to write a customized reader (internally it's leveraging S3 API to fetch a range of bytes) and passed it to function  `file.NewParquetReader()`. 
   
   Another approach is to send S3 API to fetch the last 8 bytes of parquet file to get the footer, metadata first, and then send S3 APIs to read each row group to get data using `file.NewPageReader()`.
   
   ### Component(s)
   
   Go


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org