You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/09/16 18:37:42 UTC
[GitHub] [iceberg] pavibhai opened a new issue, #5777: ORCFileAppender should fetch the stripe offsets from the Writer instead of the opening the file
pavibhai opened a new issue, #5777:
URL: https://github.com/apache/iceberg/issues/5777
### Feature Request / Improvement
Currently the ORCFileAppender opens a written ORC file just to derive the Stripe offsets
```java
@Override
public List<Long> splitOffsets() {
Preconditions.checkState(isClosed, "File is not yet closed");
try (Reader reader = ORC.newFileReader(file.toInputFile(), conf)) {
List<StripeInformation> stripes = reader.getStripes();
return Collections.unmodifiableList(Lists.transform(stripes, StripeInformation::getOffset));
} catch (IOException e) {
throw new RuntimeIOException(e, "Can't close ORC reader %s", file.location());
}
}
```
Starting with ORC 1.7 we have added a public API to retrieve offset information from the writer. This when called after close gives the complete stripe information that is written out to the file. With this we can avoid opening the written file.
### Query engine
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue closed issue #5777: ORCFileAppender should fetch the stripe offsets from the Writer instead of opening the written file
Posted by GitBox <gi...@apache.org>.
rdblue closed issue #5777: ORCFileAppender should fetch the stripe offsets from the Writer instead of opening the written file
URL: https://github.com/apache/iceberg/issues/5777
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org