You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2022/09/06 16:49:13 UTC
[GitHub] [hive] ayushtkn opened a new pull request, #3577: HIVE-26519: Iceberg: Add support for CTLT queries.
ayushtkn opened a new pull request, #3577:
URL: https://github.com/apache/hive/pull/3577
### What changes were proposed in this pull request?
Add support for ctlt queries for Iceberg tables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn commented on a diff in pull request #3577: HIVE-26519: Iceberg: Add support for CTLT queries.
Posted by GitBox <gi...@apache.org>.
ayushtkn commented on code in PR #3577:
URL: https://github.com/apache/hive/pull/3577#discussion_r965991159
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java:
##########
@@ -48,16 +48,30 @@ public class StorageFormat {
private final Map<String, String> serdeProps;
private enum StorageHandlerTypes {
- ICEBERG("\'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler\'");
+ ICEBERG("\'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler\'",
+ "org.apache.iceberg.mr.hive.HiveIcebergInputFormat", "org.apache.iceberg.mr.hive.HiveIcebergOutputFormat");
private final String className;
- StorageHandlerTypes(String className) {
+ private final String inputFormat;
+ private final String outputFormat;
+
+ StorageHandlerTypes(String className, String inputFormat, String outputFormat) {
Review Comment:
Done
##########
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java:
##########
@@ -1576,6 +1576,20 @@ public void testAlterTableToIcebergAndMetadataLocation() throws IOException {
});
}
+ @Test
+ public void testCTLT() {
+ // Create a normal table and add some data
+ shell.executeStatement("CREATE TABLE source(a int)");
+ shell.executeStatement("insert into source values(1)");
+
+ shell.executeStatement(String.format("CREATE TABLE dest LIKE source STORED BY ICEBERG %s ",
+ testTables.locationForCreateTableSQL(TableIdentifier.of("default", "dest"))));
+
+ // Try a select query and check if the table is empty .
+ String result = shell.executeAndStringify("select a from " + TableIdentifier.of("default", "dest").name());
Review Comment:
Added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3577: HIVE-26519: Iceberg: Add support for CTLT queries.
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3577:
URL: https://github.com/apache/hive/pull/3577#issuecomment-1241257617
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3577)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3577&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3577&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3577&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=CODE_SMELL) [4 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3577&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3577&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3577: HIVE-26519: Iceberg: Add support for CTLT queries.
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3577:
URL: https://github.com/apache/hive/pull/3577#issuecomment-1241362829
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3577)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3577&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3577&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3577&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=CODE_SMELL) [4 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3577&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3577&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3577: HIVE-26519: Iceberg: Add support for CTLT queries.
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3577:
URL: https://github.com/apache/hive/pull/3577#issuecomment-1238533140
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3577)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3577&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3577&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3577&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3577&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3577&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3577: HIVE-26519: Iceberg: Add support for CTLT queries.
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3577:
URL: https://github.com/apache/hive/pull/3577#issuecomment-1240857146
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3577)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3577&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3577&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3577&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=CODE_SMELL) [33 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3577&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3577&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3577&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn merged pull request #3577: HIVE-26519: Iceberg: Add support for CTLT queries.
Posted by GitBox <gi...@apache.org>.
ayushtkn merged PR #3577:
URL: https://github.com/apache/hive/pull/3577
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] lcspinter commented on a diff in pull request #3577: HIVE-26519: Iceberg: Add support for CTLT queries.
Posted by GitBox <gi...@apache.org>.
lcspinter commented on code in PR #3577:
URL: https://github.com/apache/hive/pull/3577#discussion_r964513432
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java:
##########
@@ -48,16 +48,30 @@ public class StorageFormat {
private final Map<String, String> serdeProps;
private enum StorageHandlerTypes {
- ICEBERG("\'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler\'");
+ ICEBERG("\'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler\'",
+ "org.apache.iceberg.mr.hive.HiveIcebergInputFormat", "org.apache.iceberg.mr.hive.HiveIcebergOutputFormat");
private final String className;
- StorageHandlerTypes(String className) {
+ private final String inputFormat;
+ private final String outputFormat;
+
+ StorageHandlerTypes(String className, String inputFormat, String outputFormat) {
Review Comment:
This can be private.
##########
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java:
##########
@@ -1576,6 +1576,20 @@ public void testAlterTableToIcebergAndMetadataLocation() throws IOException {
});
}
+ @Test
+ public void testCTLT() {
+ // Create a normal table and add some data
+ shell.executeStatement("CREATE TABLE source(a int)");
+ shell.executeStatement("insert into source values(1)");
+
+ shell.executeStatement(String.format("CREATE TABLE dest LIKE source STORED BY ICEBERG %s ",
+ testTables.locationForCreateTableSQL(TableIdentifier.of("default", "dest"))));
+
+ // Try a select query and check if the table is empty .
+ String result = shell.executeAndStringify("select a from " + TableIdentifier.of("default", "dest").name());
Review Comment:
Could you please validate that the table is iceberg? Also, I would be interested in checking some table properties.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org