You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by "kokila-19 (via GitHub)" <gi...@apache.org> on 2026/07/24 11:52:22 UTC
[PR] Fix Build and Publish docker images in Actions after Iceberg Upgrade [hive]
kokila-19 opened a new pull request, #6638:
URL: https://github.com/apache/hive/pull/6638
### What changes were proposed in this pull request?
Enable **metastore.iceberg.catalog.unique.table.location=true** in the docker HMS configMap.yaml for test createTableInUniqueLocation.
Fix the missing-metadata test to work with S3 (use Hadoop rename instead of local file move) for testLoadTableWithMissingMetadataFile.
### Why are the changes needed?
Two docker tests were failing:
**createTableInUniqueLocation**: HMS in docker didn’t have unique locations enabled.
**testLoadTableWithMissingMetadataFile**: the iceberg's test uses local file moves but docker stores data on S3.
### Does this PR introduce _any_ user-facing change?
No. Test and docker config only.
### How was this patch tested?
Workflow in my fork. Run Junit executes successfully
https://github.com/kokila-19/hive/actions/runs/30070902634/job/89411368628
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix Build and Publish docker images in Actions after Iceberg Upgrade [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #6638:
URL: https://github.com/apache/hive/pull/6638#issuecomment-5109974711
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6638) **Quality Gate passed**
Issues
 [34 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6638&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6638&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=6638&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6638&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6638&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6638)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix Build and Publish docker images in Actions after Iceberg Upgrade [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #6638:
URL: https://github.com/apache/hive/pull/6638#discussion_r3656541877
##########
itests/test-docker/src/test/java/org/apache/iceberg/rest/HiveIcebergRESTCatalogTests.java:
##########
@@ -75,4 +84,41 @@ protected boolean supportsNamesWithSlashes() {
protected boolean supportsServerSideRetry() {
return true;
}
+
+ /**
+ * Checks that loading a table fails clearly when its metadata file is missing.
+ *
+ * <p>The base Iceberg test moves the metadata file with {@code Files.move()} which only works
+ * on a local disk. Docker tests store data on S3, so this override renames the metadata file
+ * with Hadoop {@link FileSystem} instead, then puts it back in {@code finally}.
+ */
+ @Test
+ @Override
+ public void testLoadTableWithMissingMetadataFile(@TempDir java.nio.file.Path tempDir)
+ throws IOException {
+ RESTCatalog catalog = catalog();
+
+ if (requiresNamespaceCreate()) {
+ catalog.createNamespace(TBL.namespace());
+ }
+
+ catalog.buildTable(TBL, SCHEMA).create();
+ assertThat(catalog.tableExists(TBL)).as("Table should exist").isTrue();
+
+ Table table = catalog.loadTable(TBL);
+ String metadataFileLocation =
+ ((HasTableOperations) table).operations().current().metadataFileLocation();
+ Path metadataPath = new Path(metadataFileLocation);
+ Path backupPath = metadataPath.suffix(".bak");
+ FileSystem fs = metadataPath.getFileSystem(client.getHadoopConf());
+
+ try {
+ assertThat(fs.rename(metadataPath, backupPath)).isTrue();
Review Comment:
what is this double rename thingy?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix Build and Publish docker images in Actions after Iceberg Upgrade [hive]
Posted by "kokila-19 (via GitHub)" <gi...@apache.org>.
kokila-19 commented on code in PR #6638:
URL: https://github.com/apache/hive/pull/6638#discussion_r3667209710
##########
itests/test-docker/src/test/java/org/apache/iceberg/rest/HiveIcebergRESTCatalogTests.java:
##########
@@ -75,4 +84,41 @@ protected boolean supportsNamesWithSlashes() {
protected boolean supportsServerSideRetry() {
return true;
}
+
+ /**
+ * Checks that loading a table fails clearly when its metadata file is missing.
+ *
+ * <p>The base Iceberg test moves the metadata file with {@code Files.move()} which only works
+ * on a local disk. Docker tests store data on S3, so this override renames the metadata file
+ * with Hadoop {@link FileSystem} instead, then puts it back in {@code finally}.
+ */
+ @Test
+ @Override
+ public void testLoadTableWithMissingMetadataFile(@TempDir java.nio.file.Path tempDir)
+ throws IOException {
+ RESTCatalog catalog = catalog();
+
+ if (requiresNamespaceCreate()) {
+ catalog.createNamespace(TBL.namespace());
+ }
+
+ catalog.buildTable(TBL, SCHEMA).create();
+ assertThat(catalog.tableExists(TBL)).as("Table should exist").isTrue();
+
+ Table table = catalog.loadTable(TBL);
+ String metadataFileLocation =
+ ((HasTableOperations) table).operations().current().metadataFileLocation();
+ Path metadataPath = new Path(metadataFileLocation);
+ Path backupPath = metadataPath.suffix(".bak");
+ FileSystem fs = metadataPath.getFileSystem(client.getHadoopConf());
+
+ try {
+ assertThat(fs.rename(metadataPath, backupPath)).isTrue();
Review Comment:
Updated the test case using delete file and also tested the job in my fork
https://github.com/kokila-19/hive/actions/runs/30360512626/job/90278861651
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29764: Fix Build and Publish docker images in Actions after Iceberg Upgrade [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ merged PR #6638:
URL: https://github.com/apache/hive/pull/6638
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix Build and Publish docker images in Actions after Iceberg Upgrade [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #6638:
URL: https://github.com/apache/hive/pull/6638#issuecomment-5070356160
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6638) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6638&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6638&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=6638&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6638&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6638&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6638)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix Build and Publish docker images in Actions after Iceberg Upgrade [hive]
Posted by "kokila-19 (via GitHub)" <gi...@apache.org>.
kokila-19 commented on PR #6638:
URL: https://github.com/apache/hive/pull/6638#issuecomment-5087707650
@deniskuzZ Could you please review this ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix Build and Publish docker images in Actions after Iceberg Upgrade [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #6638:
URL: https://github.com/apache/hive/pull/6638#discussion_r3656541877
##########
itests/test-docker/src/test/java/org/apache/iceberg/rest/HiveIcebergRESTCatalogTests.java:
##########
@@ -75,4 +84,41 @@ protected boolean supportsNamesWithSlashes() {
protected boolean supportsServerSideRetry() {
return true;
}
+
+ /**
+ * Checks that loading a table fails clearly when its metadata file is missing.
+ *
+ * <p>The base Iceberg test moves the metadata file with {@code Files.move()} which only works
+ * on a local disk. Docker tests store data on S3, so this override renames the metadata file
+ * with Hadoop {@link FileSystem} instead, then puts it back in {@code finally}.
+ */
+ @Test
+ @Override
+ public void testLoadTableWithMissingMetadataFile(@TempDir java.nio.file.Path tempDir)
+ throws IOException {
+ RESTCatalog catalog = catalog();
+
+ if (requiresNamespaceCreate()) {
+ catalog.createNamespace(TBL.namespace());
+ }
+
+ catalog.buildTable(TBL, SCHEMA).create();
+ assertThat(catalog.tableExists(TBL)).as("Table should exist").isTrue();
+
+ Table table = catalog.loadTable(TBL);
+ String metadataFileLocation =
+ ((HasTableOperations) table).operations().current().metadataFileLocation();
+ Path metadataPath = new Path(metadataFileLocation);
+ Path backupPath = metadataPath.suffix(".bak");
+ FileSystem fs = metadataPath.getFileSystem(client.getHadoopConf());
+
+ try {
+ assertThat(fs.rename(metadataPath, backupPath)).isTrue();
Review Comment:
what is this double rename thingy?
````
table.io().deleteFile(metadataFileLocation);
````
doesn't work as expected?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29764: Fix Build and Publish docker images in Actions after Iceberg Upgrade [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on PR #6638:
URL: https://github.com/apache/hive/pull/6638#issuecomment-5124900644
Nice. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org