[GitHub] [iceberg] kbendick commented on issue #3062: Report I/O metrics to Spark

[GitBox <gi...@apache.org>]

kbendick commented on issue #3062:
URL: https://github.com/apache/iceberg/issues/3062#issuecomment-983250944


   Thank you for the report @fcvr1010 ans @kishendas!
   
   This is not unexpected behavior, depending on which FileIO implementation you're using. I believe S3FileIO doesnt emit the metrics but if one uses HDFS it does.
   
   I am a bit surprised it doesn't come up on databricks but that's only based off of a hunch (not concrete experience). If I could talk to you a bit more @kishendas that would be appreciated!
   
   There is a push for more standardization around metrics coming soon (though I'm not sure when exactly). Specifically to deal with issues like this.
   
   I'm not exactly sure how easily this specific issue can be addressed as I haven't looked into the details much. Could be a simple fix and might not be.
   
   Perhaps others will have more insight than I can provide.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org