[GitHub] [iceberg] steveloughran commented on pull request #7114: Build: Move to Hadoop3

["steveloughran (via GitHub)" <gi...@apache.org>]

steveloughran commented on PR #7114:
URL: https://github.com/apache/iceberg/pull/7114#issuecomment-1508328430

   so this sets the minimum version iceberg builds against to being 3.3.4? nice. 
   
   with that you can use openFile(path) and declare the read options you want (sequential vs random), any split start/end and, if you know the length, pass that down to save on any HEAD request. s3a and gcs connectors like this
   
   The other bit of fun 3.3.4 has is the whole IOStatistics API to get stats collected on filesystem input and output streams, filesystem instances and others, with
   the ability to take a snapshot (IOStatisticsSnapshot) which can be serialized as java or json. 
   
   You can just call .toString() of streams to see what is collected, and/or set this to be logged in filesystem close, which is an easy way to see what is happening in existing code
   
   ```xml
     <property>
       <name>fs.iostatistics.logging.level</name>
       <value>info</value>
     </property>
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org