You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "Fokko (via GitHub)" <gi...@apache.org> on 2023/03/15 12:23:32 UTC
[GitHub] [iceberg] Fokko opened a new pull request, #7114: Build: Move to Hadoop3
Fokko opened a new pull request, #7114:
URL: https://github.com/apache/iceberg/pull/7114
I noticed that we use Hadoop3 as a library in https://github.com/tabular-io/iceberg-rest-image. So I think it makes sense to compile against this version as well at some point. Including the latest version of Hive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko closed pull request #7114: Build: Move to Hadoop3
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko closed pull request #7114: Build: Move to Hadoop3
URL: https://github.com/apache/iceberg/pull/7114
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on pull request #7114: Build: Move to Hadoop3
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on PR #7114:
URL: https://github.com/apache/iceberg/pull/7114#issuecomment-1514269770
Closing as it is a duplicate of https://github.com/apache/iceberg/pull/5024
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] steveloughran commented on pull request #7114: Build: Move to Hadoop3
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on PR #7114:
URL: https://github.com/apache/iceberg/pull/7114#issuecomment-1508328430
so this sets the minimum version iceberg builds against to being 3.3.4? nice.
with that you can use openFile(path) and declare the read options you want (sequential vs random), any split start/end and, if you know the length, pass that down to save on any HEAD request. s3a and gcs connectors like this
The other bit of fun 3.3.4 has is the whole IOStatistics API to get stats collected on filesystem input and output streams, filesystem instances and others, with
the ability to take a snapshot (IOStatisticsSnapshot) which can be serialized as java or json.
You can just call .toString() of streams to see what is collected, and/or set this to be logged in filesystem close, which is an easy way to see what is happening in existing code
```xml
<property>
<name>fs.iostatistics.logging.level</name>
<value>info</value>
</property>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org