You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "coded9 (via GitHub)" <gi...@apache.org> on 2025/05/24 07:56:12 UTC
[PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
coded9 opened a new pull request, #13143:
URL: https://github.com/apache/iceberg/pull/13143
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "coded9 (via GitHub)" <gi...@apache.org>.
coded9 commented on code in PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#discussion_r2450451901
##########
core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java:
##########
@@ -959,6 +966,35 @@ public void testLoadMissingTable() {
.hasMessageStartingWith("Table does not exist: ns.tbl");
}
+ @Test
+ public void testLoadTableWithMissingMetadatafile(@TempDir Path tempDir) throws IOException {
Review Comment:
Yes, the ErrorHandlers is specific to REST catalog, but the test would be applicable for all kind of catalogs. All other catalogs return NotFound exception
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#issuecomment-3679785952
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "coded9 (via GitHub)" <gi...@apache.org>.
coded9 commented on PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#issuecomment-3713054941
> @coded9 thanks for the change, and sorry for the delay. LGTM. Could resolve the conflicts?
Resolved.Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "coded9 (via GitHub)" <gi...@apache.org>.
coded9 commented on code in PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#discussion_r2159971727
##########
core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java:
##########
@@ -959,6 +959,16 @@ public void testLoadMissingTable() {
.hasMessageStartingWith("Table does not exist: ns.tbl");
}
+ @Test
+ public void testLoadTableWithMissingMetadatafile() {
+ C catalog = catalog();
+
+ assertThat(catalog.tableExists(TBL)).as("Table should not exist").isFalse();
+ assertThatThrownBy(() -> catalog.loadTable(TBL))
+ .isInstanceOf(NoSuchTableException.class)
Review Comment:
Fixed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#issuecomment-3720239208
Hi @coded9 , I will merge it by EOD if there is no objection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain merged PR #13143:
URL: https://github.com/apache/iceberg/pull/13143
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#discussion_r2112280095
##########
core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java:
##########
@@ -959,6 +959,16 @@ public void testLoadMissingTable() {
.hasMessageStartingWith("Table does not exist: ns.tbl");
}
+ @Test
+ public void testLoadTableWithMissingMetadatafile() {
+ C catalog = catalog();
+
+ assertThat(catalog.tableExists(TBL)).as("Table should not exist").isFalse();
+ assertThatThrownBy(() -> catalog.loadTable(TBL))
+ .isInstanceOf(NoSuchTableException.class)
Review Comment:
the test doesn't reproduce the issue you're trying to repro and it passes without applying the fix that is being proposed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#discussion_r2364803142
##########
core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java:
##########
@@ -959,6 +966,35 @@ public void testLoadMissingTable() {
.hasMessageStartingWith("Table does not exist: ns.tbl");
}
+ @Test
+ public void testLoadTableWithMissingMetadatafile(@TempDir Path tempDir) throws IOException {
Review Comment:
Nit: -> `testLoadTableWithMissingMetadataFile`
##########
core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java:
##########
@@ -959,6 +966,35 @@ public void testLoadMissingTable() {
.hasMessageStartingWith("Table does not exist: ns.tbl");
}
+ @Test
+ public void testLoadTableWithMissingMetadatafile(@TempDir Path tempDir) throws IOException {
Review Comment:
`CatalogTests` is extended by multiple catalog suites. I think the change in class `ErrorHanlders` only applies to REST catalog. We may not need it to be triggered in non-rest catalog, like `TestHiveCatalog`, `TestBigQueryCatalog`
##########
core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java:
##########
@@ -959,6 +966,35 @@ public void testLoadMissingTable() {
.hasMessageStartingWith("Table does not exist: ns.tbl");
}
+ @Test
+ public void testLoadTableWithMissingMetadatafile(@TempDir Path tempDir) throws IOException {
+ C catalog = catalog();
+
+ if (requiresNamespaceCreate()) {
+ catalog.createNamespace(TBL.namespace());
+ }
+
+ catalog.buildTable(TBL, SCHEMA).create();
+ assertThat(catalog.tableExists(TBL)).as("Table should exist").isTrue();
+
+ Table table = catalog.loadTable(TBL);
+ String metadataFileLocation =
+ ((HasTableOperations) table).operations().current().metadataFileLocation();
+ Path renamedMetadataFile = tempDir.resolve("tmp.json");
+ renamedMetadataFile.toFile().deleteOnExit();
+ Files.writeString(renamedMetadataFile, "metadata");
+ Path metadataFilePath =
+ metadataFileLocation.startsWith("file:")
+ ? Paths.get(URI.create(metadataFileLocation))
+ : Paths.get(metadataFileLocation);
+ Files.move(metadataFilePath, renamedMetadataFile, StandardCopyOption.REPLACE_EXISTING);
+
+ assertThatThrownBy(() -> catalog.loadTable(TBL))
+ .isInstanceOf(NotFoundException.class)
+ .hasMessageContaining("Failed to open input stream for file: " + metadataFileLocation);
+ Files.move(renamedMetadataFile, metadataFilePath, StandardCopyOption.REPLACE_EXISTING);
Review Comment:
Should we wrap the recovery logic in a `try/finally` block?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "coded9 (via GitHub)" <gi...@apache.org>.
coded9 commented on PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#issuecomment-3680421144
@flyrain Bumping for review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#issuecomment-3420095133
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "coded9 (via GitHub)" <gi...@apache.org>.
coded9 commented on code in PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#discussion_r2450450779
##########
core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java:
##########
@@ -959,6 +966,35 @@ public void testLoadMissingTable() {
.hasMessageStartingWith("Table does not exist: ns.tbl");
}
+ @Test
+ public void testLoadTableWithMissingMetadatafile(@TempDir Path tempDir) throws IOException {
Review Comment:
Addressed
##########
core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java:
##########
@@ -959,6 +966,35 @@ public void testLoadMissingTable() {
.hasMessageStartingWith("Table does not exist: ns.tbl");
}
+ @Test
+ public void testLoadTableWithMissingMetadatafile(@TempDir Path tempDir) throws IOException {
+ C catalog = catalog();
+
+ if (requiresNamespaceCreate()) {
+ catalog.createNamespace(TBL.namespace());
+ }
+
+ catalog.buildTable(TBL, SCHEMA).create();
+ assertThat(catalog.tableExists(TBL)).as("Table should exist").isTrue();
+
+ Table table = catalog.loadTable(TBL);
+ String metadataFileLocation =
+ ((HasTableOperations) table).operations().current().metadataFileLocation();
+ Path renamedMetadataFile = tempDir.resolve("tmp.json");
+ renamedMetadataFile.toFile().deleteOnExit();
+ Files.writeString(renamedMetadataFile, "metadata");
+ Path metadataFilePath =
+ metadataFileLocation.startsWith("file:")
+ ? Paths.get(URI.create(metadataFileLocation))
+ : Paths.get(metadataFileLocation);
+ Files.move(metadataFilePath, renamedMetadataFile, StandardCopyOption.REPLACE_EXISTING);
+
+ assertThatThrownBy(() -> catalog.loadTable(TBL))
+ .isInstanceOf(NotFoundException.class)
+ .hasMessageContaining("Failed to open input stream for file: " + metadataFileLocation);
+ Files.move(renamedMetadataFile, metadataFilePath, StandardCopyOption.REPLACE_EXISTING);
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "coded9 (via GitHub)" <gi...@apache.org>.
coded9 commented on PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#issuecomment-3718837541
@flyrain Can we merge this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#issuecomment-3240534682
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "coded9 (via GitHub)" <gi...@apache.org>.
coded9 commented on PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#issuecomment-3520540243
@flyrain could you review?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "coded9 (via GitHub)" <gi...@apache.org>.
coded9 commented on PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#issuecomment-3240718598
Will be raising a proposal on the mailing list
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "coded9 (via GitHub)" <gi...@apache.org>.
coded9 commented on PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#issuecomment-3562219137
@flyrain Bumping for review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on PR #13143:
URL: https://github.com/apache/iceberg/pull/13143#issuecomment-3725465869
There was a post-commit failure here, https://github.com/apache/iceberg/actions/runs/20802340622. Doesn't seem to be related.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org