You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "prakhar10 (via GitHub)" <gi...@apache.org> on 2025/07/30 18:30:57 UTC
[PR] Strip trailing slash from default warehouse location [iceberg]
prakhar10 opened a new pull request, #13710:
URL: https://github.com/apache/iceberg/pull/13710
With native s3 filesystem support in Trino 458 and above, its unable to find the metadata s3 location with // and fails the query with location not found error.
This PR strips the trailing slash from the warehouse location to fix this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Strip trailing slash from default warehouse location [iceberg]
Posted by "dramaticlly (via GitHub)" <gi...@apache.org>.
dramaticlly commented on code in PR #13710:
URL: https://github.com/apache/iceberg/pull/13710#discussion_r2243925525
##########
hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java:
##########
@@ -707,7 +707,8 @@ protected String defaultWarehouseLocation(TableIdentifier tableIdentifier) {
clients.run(client -> client.getDatabase(tableIdentifier.namespace().levels()[0]));
if (databaseData.getLocationUri() != null) {
// If the database location is set use it as a base.
- return String.format("%s/%s", databaseData.getLocationUri(), tableIdentifier.name());
+ String stripLocationUri = LocationUtil.stripTrailingSlash(databaseData.getLocationUri());
+ return String.format("%s/%s", stripLocationUri, tableIdentifier.name());
Review Comment:
@prakhar10 can you add a corresponding unit test to illustrate the failure before and success after this change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Strip trailing slash from default warehouse location [iceberg]
Posted by "prakhar10 (via GitHub)" <gi...@apache.org>.
prakhar10 commented on PR #13710:
URL: https://github.com/apache/iceberg/pull/13710#issuecomment-3141441026
> @prakhar10 Do you understand Trino doesn't depend on the `HiveCatalog` class?
ok, I was not aware of that. We tried to strip the extra slash from the warehouse location and it worked with Trino native s3 filesystem. So I created this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Strip trailing slash from default warehouse location [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13710:
URL: https://github.com/apache/iceberg/pull/13710#issuecomment-3300745881
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Strip trailing slash from default warehouse location [iceberg]
Posted by "dramaticlly (via GitHub)" <gi...@apache.org>.
dramaticlly commented on code in PR #13710:
URL: https://github.com/apache/iceberg/pull/13710#discussion_r2243925525
##########
hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java:
##########
@@ -707,7 +707,8 @@ protected String defaultWarehouseLocation(TableIdentifier tableIdentifier) {
clients.run(client -> client.getDatabase(tableIdentifier.namespace().levels()[0]));
if (databaseData.getLocationUri() != null) {
// If the database location is set use it as a base.
- return String.format("%s/%s", databaseData.getLocationUri(), tableIdentifier.name());
+ String stripLocationUri = LocationUtil.stripTrailingSlash(databaseData.getLocationUri());
+ return String.format("%s/%s", stripLocationUri, tableIdentifier.name());
Review Comment:
@prakhar10 can you add a corresponding unit test to illustrate the failure before and succeeded after the change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Strip trailing slash from default warehouse location [iceberg]
Posted by "danielcweeks (via GitHub)" <gi...@apache.org>.
danielcweeks commented on code in PR #13710:
URL: https://github.com/apache/iceberg/pull/13710#discussion_r2245804479
##########
hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java:
##########
@@ -707,7 +707,8 @@ protected String defaultWarehouseLocation(TableIdentifier tableIdentifier) {
clients.run(client -> client.getDatabase(tableIdentifier.namespace().levels()[0]));
if (databaseData.getLocationUri() != null) {
Review Comment:
You might want to change this to `Strings.isNullOrEmpty` otherwise you have a mismatch between this check and the one performed by `LocationUtil::stripTriailingSlash`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Strip trailing slash from default warehouse location [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #13710: Strip trailing slash from default warehouse location
URL: https://github.com/apache/iceberg/pull/13710
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Strip trailing slash from default warehouse location [iceberg]
Posted by "prakhar10 (via GitHub)" <gi...@apache.org>.
prakhar10 commented on code in PR #13710:
URL: https://github.com/apache/iceberg/pull/13710#discussion_r2246449604
##########
hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java:
##########
@@ -707,7 +707,8 @@ protected String defaultWarehouseLocation(TableIdentifier tableIdentifier) {
clients.run(client -> client.getDatabase(tableIdentifier.namespace().levels()[0]));
if (databaseData.getLocationUri() != null) {
// If the database location is set use it as a base.
- return String.format("%s/%s", databaseData.getLocationUri(), tableIdentifier.name());
+ String stripLocationUri = LocationUtil.stripTrailingSlash(databaseData.getLocationUri());
+ return String.format("%s/%s", stripLocationUri, tableIdentifier.name());
Review Comment:
Hi @dramaticlly , so we came across this issue when we create a table using spark-iceberg which adds `//` to the location and when we access the table using Trino 458+ version with native s3 filesystem support.
I'm not really sure how would I be able to right a unit test to show the failure using Trino. If there are already any similar examples can you point me to them please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Strip trailing slash from default warehouse location [iceberg]
Posted by "ebyhr (via GitHub)" <gi...@apache.org>.
ebyhr commented on PR #13710:
URL: https://github.com/apache/iceberg/pull/13710#issuecomment-3141431202
@prakhar10 Do you understand Trino doesn't depend on the `HiveCatalog` class?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Strip trailing slash from default warehouse location [iceberg]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #13710:
URL: https://github.com/apache/iceberg/pull/13710#discussion_r2264929520
##########
hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java:
##########
@@ -707,7 +707,8 @@ protected String defaultWarehouseLocation(TableIdentifier tableIdentifier) {
clients.run(client -> client.getDatabase(tableIdentifier.namespace().levels()[0]));
if (databaseData.getLocationUri() != null) {
// If the database location is set use it as a base.
- return String.format("%s/%s", databaseData.getLocationUri(), tableIdentifier.name());
+ String stripLocationUri = LocationUtil.stripTrailingSlash(databaseData.getLocationUri());
+ return String.format("%s/%s", stripLocationUri, tableIdentifier.name());
Review Comment:
maybe add something like [this](https://github.com/apache/iceberg/blob/main/aws/src/test/java/org/apache/iceberg/aws/glue/TestGlueCatalog.java#L134-L142) to https://github.com/apache/iceberg/blob/c8c61a3f6b60499f09e5047c5c9d8a3c02d9fb72/hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java#L95
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Strip trailing slash from default warehouse location [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13710:
URL: https://github.com/apache/iceberg/pull/13710#issuecomment-3268423271
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org