You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2020/02/17 04:37:01 UTC
[GitHub] [incubator-iceberg] waterlx opened a new pull request #806: Fix NPE
when calling locationProvider() in HadoopTableOperations with no current
metadata
waterlx opened a new pull request #806: Fix NPE when calling locationProvider() in HadoopTableOperations with no current metadata
URL: https://github.com/apache/incubator-iceberg/pull/806
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [incubator-iceberg] waterlx commented on issue #806: Fix NPE when
calling locationProvider() in HadoopTableOperations with no current metadata
Posted by GitBox <gi...@apache.org>.
waterlx commented on issue #806: Fix NPE when calling locationProvider() in HadoopTableOperations with no current metadata
URL: https://github.com/apache/incubator-iceberg/pull/806#issuecomment-586817030
The fix is a little weird to me as it is not fixed in HadoopTableOperations directly, but in BaseTransaction.TransactionTable, the caller of HadoopTableOperations#locationProvider(), because I could not find properties in HadoopTableOperations to pass to LocationProviders.locationFor() when current() returns null(no metadata at all).
Please kindly let me know if there is a better way to do that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [incubator-iceberg] waterlx commented on issue #806: Fix NPE when
calling locationProvider() in HadoopTableOperations with no current metadata
Posted by GitBox <gi...@apache.org>.
waterlx commented on issue #806: Fix NPE when calling locationProvider() in HadoopTableOperations with no current metadata
URL: https://github.com/apache/incubator-iceberg/pull/806#issuecomment-587533734
Hi @rdblue Thanks for the guide!
The PR is updated based on your comment and with a UT added. Please help to review it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [incubator-iceberg] rdblue commented on issue #806: Fix NPE when
calling locationProvider() in HadoopTableOperations with no current metadata
Posted by GitBox <gi...@apache.org>.
rdblue commented on issue #806: Fix NPE when calling locationProvider() in HadoopTableOperations with no current metadata
URL: https://github.com/apache/incubator-iceberg/pull/806#issuecomment-587582411
Thanks @waterlx! Merged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [incubator-iceberg] waterlx commented on issue #806: Fix NPE when
calling locationProvider() in HadoopTableOperations with no current metadata
Posted by GitBox <gi...@apache.org>.
waterlx commented on issue #806: Fix NPE when calling locationProvider() in HadoopTableOperations with no current metadata
URL: https://github.com/apache/incubator-iceberg/pull/806#issuecomment-586817260
@rdblue @chenjunjiedada would you please help to review and kindly comment?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [incubator-iceberg] rdblue merged pull request #806: Fix NPE when
calling locationProvider() in HadoopTableOperations with no current metadata
Posted by GitBox <gi...@apache.org>.
rdblue merged pull request #806: Fix NPE when calling locationProvider() in HadoopTableOperations with no current metadata
URL: https://github.com/apache/incubator-iceberg/pull/806
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [incubator-iceberg] rdblue commented on issue #806: Fix NPE when
calling locationProvider() in HadoopTableOperations with no current metadata
Posted by GitBox <gi...@apache.org>.
rdblue commented on issue #806: Fix NPE when calling locationProvider() in HadoopTableOperations with no current metadata
URL: https://github.com/apache/incubator-iceberg/pull/806#issuecomment-587185576
@waterlx, the problem is that `HadoopTableOperations` doesn't implement the optional `temp(TableMetadata)` method. That method is used to get a temporary `HadoopTableOperations` object for calls like `locationProvider()` that use table metadata. It is used by the `Transaction` system.
Previously, Hadoop tables weren't used with the transactions created by `BaseMetastoreCatalog`, so we didn't have an implementation. Now that there is a `HadoopCatalog`, it needs this to avoid the problem you're hitting.
Here's a quick implementation that I think should work:
```java
@Override
public TableOperations temp(TableMetadata uncommittedMetadata) {
return new TableOperations() {
@Override
public TableMetadata current() {
return uncommittedMetadata;
}
@Override
public TableMetadata refresh() {
throw new UnsupportedOperationException("Cannot call refresh on temporary table operations");
}
@Override
public void commit(TableMetadata base, TableMetadata metadata) {
throw new UnsupportedOperationException("Cannot call commit on temporary table operations");
}
@Override
public String metadataFileLocation(String fileName) {
return HadoopTableOperations.this.metadataFileLocation(fileName);
}
@Override
public LocationProvider locationProvider() {
return LocationProviders.locationsFor(uncommittedMetadata.location(), uncommittedMetadata.properties());
}
@Override
public FileIO io() {
return HadoopTableOperations.this.io();
}
@Override
public EncryptionManager encryption() {
return HadoopTableOperations.this.encryption();
}
@Override
public long newSnapshotId() {
return HadoopTableOperations.this.newSnapshotId();
}
};
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org