You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by "zhangbutao (via GitHub)" <gi...@apache.org> on 2023/05/04 14:48:16 UTC
[GitHub] [hive] zhangbutao opened a new pull request, #4292: [WIP]HIVE-27302: Iceberg: Suport write to iceberg branch
zhangbutao opened a new pull request, #4292:
URL: https://github.com/apache/hive/pull/4292
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/Hive/HowToContribute
2. Ensure that you have created an issue on the Hive project JIRA: https://issues.apache.org/jira/projects/HIVE/summary
3. Ensure you have added or run the appropriate tests for your PR:
4. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP]HIVE-XXXXX: Your PR title ...'.
5. Be sure to keep the PR description updated to reflect all changes.
6. Please write your PR title to summarize what this PR proposes.
7. If possible, provide a concise example to reproduce the issue for a faster review.
-->
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
### Does this PR introduce _any_ user-facing change?
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description, screenshot and/or a reproducable example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released Hive versions or within the unreleased branches such as master.
If no, write 'No'.
-->
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao closed pull request #4292: [WIP]HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao closed pull request #4292: [WIP]HIVE-27302: Iceberg: Suport write to iceberg branch
URL: https://github.com/apache/hive/pull/4292
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1201921575
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:
##########
@@ -8006,12 +8012,12 @@ && enableColumnStatsCollecting()
&& ColumnStatsAutoGatherContext.canRunAutogatherStats(fso)) {
if (destType == QBMetaData.DEST_TABLE) {
genAutoColumnStatsGatheringPipeline(destinationTable, partSpec, input,
- qb.getParseInfo().isInsertIntoTable(destinationTable.getDbName(), destinationTable.getTableName()),
- false);
+ qb.getParseInfo().isInsertIntoTable(destinationTable.getDbName(), destinationTable.getTableName(),
+ destinationTable.getBranchName()),false);
} else if (destType == QBMetaData.DEST_PARTITION) {
genAutoColumnStatsGatheringPipeline(destinationTable, destinationPartition.getSpec(), input,
- qb.getParseInfo().isInsertIntoTable(destinationTable.getDbName(), destinationTable.getTableName()),
- false);
+ qb.getParseInfo().isInsertIntoTable(destinationTable.getDbName(), destinationTable.getTableName(),
+ destinationTable.getBranchName()),false);
Review Comment:
nit: missing space
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1203996128
##########
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java:
##########
@@ -763,6 +763,9 @@ public static TableDesc getTableDesc(Table tbl) {
if (tbl.getMetaTable() != null) {
props.put("metaTable", tbl.getMetaTable());
}
+ if (tbl.getBranchName() != null) {
+ props.put("branch_name", tbl.getBranchName());
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1561097305
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [9 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1562284482
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [9 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao closed pull request #4292: [WIP]HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao closed pull request #4292: [WIP]HIVE-27302: Iceberg: Suport write to iceberg branch
URL: https://github.com/apache/hive/pull/4292
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1221257910
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -550,14 +550,33 @@ default boolean isTimeTravelAllowed() {
return false;
}
+ /**
+ * Introduced by HIVE-25457 for iceberg to query metadata table.
+ * @return true if the storage handler can support it
+ * @deprecated Use {@link #isTableMetaRefSupported()}
+ */
+ @Deprecated
default boolean isMetadataTableSupported() {
return false;
}
Review Comment:
Should be
```
@Deprecated
default boolean isMetadataTableSupported() {
return isTableMetaRefSupported();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1549349919
@deniskuzZ @ayushtkn Would you like to take a look if you have a chance? Thanks in advance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1201930835
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java:
##########
@@ -211,8 +211,13 @@ public boolean isDestToOpTypeInsertOverwrite(String clause) {
/**
* See also {@link #getInsertOverwriteTables()}
*/
- public boolean isInsertIntoTable(String dbName, String table) {
- String fullName = dbName + "." + table;
+ public boolean isInsertIntoTable(String dbName, String table, String branchName) {
+ String fullName;
+ if (branchName != null) {
Review Comment:
could we have some helper for that to avoid duplication in multiple places:
see https://github.com/apache/hive/pull/4292/files#diff-d4b1a32bbbd9e283893a6b52854c7aeb3e356a1ba1add2c4107e52901ca268f9R7418-R7424
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java:
##########
@@ -211,8 +211,13 @@ public boolean isDestToOpTypeInsertOverwrite(String clause) {
/**
* See also {@link #getInsertOverwriteTables()}
*/
- public boolean isInsertIntoTable(String dbName, String table) {
- String fullName = dbName + "." + table;
+ public boolean isInsertIntoTable(String dbName, String table, String branchName) {
+ String fullName;
+ if (branchName != null) {
Review Comment:
could we have some helper for that to avoid duplication in multiple places:
see https://github.com/apache/hive/pull/4292/files#diff-d4b1a32bbbd9e283893a6b52854c7aeb3e356a1ba1add2c4107e52901ca268f9R7418-R7424
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1213922691
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java:
##########
@@ -107,6 +109,7 @@ public static String escapeString(String str) {
static final byte[] tabEscapeBytes = "\\t".getBytes();;
static final byte[] tabUnescapeBytes = "\t".getBytes();
static final byte[] ctrlABytes = "\u0001".getBytes();
+ static final Pattern BRANCH = Pattern.compile("branch_(.*)");
Review Comment:
If table name or database name has branch? Will this work?
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -550,14 +550,19 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableMetaRefSupported() {
return false;
}
Review Comment:
Isn't this an incompatible change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1578997388
@ayushtkn @deniskuzZ Can we continue with this PR? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1581506563
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [11 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1549714369
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [10 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1200765432
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##########
@@ -661,7 +666,7 @@ public boolean isTimeTravelAllowed() {
}
@Override
- public boolean isMetadataTableSupported() {
+ public boolean isTableIdentifierSupported() {
Review Comment:
not sure it's an expected rename. Metadata table is supported only in iceberg, but table id should be available for every table type
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1204364886
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java:
##########
@@ -5406,16 +5406,16 @@ protected Table getTableObjectByName(String tabName, boolean throwException) thr
String[] names = Utilities.getDbTableName(tabName);
final String tableName = names[1];
final String dbName = names[0];
- String metaTable = null;
+ String tableIdentifier = null;
Review Comment:
I finally gave up reusing `TableName.fromString` here as i found some failed tests.
There are some code snippets use `catalog.dbName.tblName` to identify a table and `TableName.fromString` can not distinguish between
`catalog.dbName.tblName` and iceberg's `dbName.tblName.metaTable`. I have no better idea to handle this now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1204071713
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##########
@@ -661,7 +666,7 @@ public boolean isTimeTravelAllowed() {
}
@Override
- public boolean isMetadataTableSupported() {
+ public boolean isTableIdentifierSupported() {
Review Comment:
I explained this in following comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1203984008
##########
common/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java:
##########
@@ -478,7 +478,7 @@ public enum ErrorMsg {
DATACONNECTOR_NOT_EXISTS(10428, "Dataconnector does not exist:"),
TIME_TRAVEL_NOT_ALLOWED(10429, "Time travel is not allowed for {0}. Please choose a storage format which supports the feature.", true),
INVALID_METADATA_TABLE_NAME(10430, "Invalid metadata table name {0}.", true),
- METADATA_TABLE_NOT_SUPPORTED(10431, "Metadata tables are not supported for table {0}.", true),
+ TABLE_IDENTIFIER_NOT_SUPPORTED(10431, "Table identifier is not supported for table {0}.", true),
Review Comment:
It essentially means the same thing that this table doesn't support three layer namespace, e.g ` dbName.tblName.metaTable` or `dbName.tblName.branch_branchName`.
The previos errors just only considered the iceberg meta table ` dbName.tblName.metaTable`. here i refined it and I want to use a general term `Table identifier` to say that this table supports three layer namespace
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: [WIP]HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1537079256
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [13 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1200757391
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergInputFormat.java:
##########
@@ -143,6 +144,8 @@ public InputSplit[] getSplits(JobConf job, int numSplits) throws IOException {
job.set(InputFormatConfig.AS_OF_TIMESTAMP, job.get(TableScanDesc.AS_OF_TIMESTAMP, "-1"));
job.set(InputFormatConfig.SNAPSHOT_ID, job.get(TableScanDesc.AS_OF_VERSION, "-1"));
job.set(InputFormatConfig.SNAPSHOT_ID_INTERVAL_FROM, job.get(TableScanDesc.FROM_VERSION, "-1"));
+ Optional.ofNullable(job.get(TableScanDesc.BRANCH_NAME)).ifPresent(branchName ->
Review Comment:
could we change to below to be consistent
````
job.set(InputFormatConfig. OUTPUT_TABLE_BRANCH, job.get(TableScanDesc. BRANCH_NAME, ""));
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1200780120
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -1751,15 +1751,22 @@ public Table getTable(final String dbName, final String tableName, String metaTa
}
Table t = new Table(tTable);
- if (metaTableName != null) {
- if (t.getStorageHandler() == null || !t.getStorageHandler().isMetadataTableSupported()) {
- throw new SemanticException(ErrorMsg.METADATA_TABLE_NOT_SUPPORTED, t.getTableName());
+ if (tableIdentifier != null) {
+ if (t.getStorageHandler() == null || !t.getStorageHandler().isTableIdentifierSupported()) {
+ throw new SemanticException(ErrorMsg.TABLE_IDENTIFIER_NOT_SUPPORTED, t.getTableName());
}
- if (!t.getStorageHandler().isValidMetadataTable(metaTableName)) {
- throw new SemanticException(ErrorMsg.INVALID_METADATA_TABLE_NAME, metaTableName);
+ if (tableIdentifier.startsWith("branch_")) {
+ if (!t.getStorageHandler().isValidBranch(tTable, tableIdentifier.substring(7))) {
+ throw new SemanticException(String.format("Cannot use branch (does not exist): %s", tableIdentifier.substring(7)));
+ }
+ t.setBranchName(tableIdentifier);
+ } else {
+ if (!t.getStorageHandler().isValidMetadataTable(tableIdentifier)) {
Review Comment:
that is very confusing due to partial rename: isValidMetadataTable <--> isTableIdentifierSupported
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1200784808
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -542,14 +542,18 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableIdentifierSupported() {
return false;
}
default boolean isValidMetadataTable(String metaTableName) {
return false;
}
+ default boolean isValidBranch(org.apache.hadoop.hive.metastore.api.Table hmsTable, String branchName) {
Review Comment:
other storageHandlers might not have branch support at all
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1204003271
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -1751,15 +1751,22 @@ public Table getTable(final String dbName, final String tableName, String metaTa
}
Table t = new Table(tTable);
- if (metaTableName != null) {
- if (t.getStorageHandler() == null || !t.getStorageHandler().isMetadataTableSupported()) {
- throw new SemanticException(ErrorMsg.METADATA_TABLE_NOT_SUPPORTED, t.getTableName());
+ if (tableIdentifier != null) {
+ if (t.getStorageHandler() == null || !t.getStorageHandler().isTableIdentifierSupported()) {
+ throw new SemanticException(ErrorMsg.TABLE_IDENTIFIER_NOT_SUPPORTED, t.getTableName());
}
- if (!t.getStorageHandler().isValidMetadataTable(metaTableName)) {
- throw new SemanticException(ErrorMsg.INVALID_METADATA_TABLE_NAME, metaTableName);
+ if (tableIdentifier.startsWith("branch_")) {
+ if (!t.getStorageHandler().isValidBranch(tTable, tableIdentifier.substring(7))) {
+ throw new SemanticException(String.format("Cannot use branch (does not exist): %s", tableIdentifier.substring(7)));
Review Comment:
Fixed. I use the added util method HiveUtils::getTableBranch to avoid this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1205352972
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -542,14 +542,18 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableIdentifierSupported() {
return false;
}
default boolean isValidMetadataTable(String metaTableName) {
return false;
}
+ default boolean isValidBranch(org.apache.hadoop.hive.metastore.api.Table hmsTable, String branchName) {
Review Comment:
what if we have generic `isValidTableMetaRef ` that could validate metaRef based on supplied type?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1211968021
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -542,14 +542,18 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableIdentifierSupported() {
return false;
}
default boolean isValidMetadataTable(String metaTableName) {
return false;
}
+ default boolean isValidBranch(org.apache.hadoop.hive.metastore.api.Table hmsTable, String branchName) {
Review Comment:
👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1217344339
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -550,14 +550,19 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableMetaRefSupported() {
return false;
}
Review Comment:
Thanks @ayushtkn for explanation. Sound reasonable. IMO, maybe we should keep the name as is `isMetadataTableSupported()` and add a new method `isTableMetaRefSupported()`? wdyt, @deniskuzZ
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: [WIP]HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1546217941
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [26 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1208398073
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -1751,15 +1751,22 @@ public Table getTable(final String dbName, final String tableName, String metaTa
}
Table t = new Table(tTable);
- if (metaTableName != null) {
- if (t.getStorageHandler() == null || !t.getStorageHandler().isMetadataTableSupported()) {
- throw new SemanticException(ErrorMsg.METADATA_TABLE_NOT_SUPPORTED, t.getTableName());
+ if (tableIdentifier != null) {
+ if (t.getStorageHandler() == null || !t.getStorageHandler().isTableIdentifierSupported()) {
+ throw new SemanticException(ErrorMsg.TABLE_IDENTIFIER_NOT_SUPPORTED, t.getTableName());
}
- if (!t.getStorageHandler().isValidMetadataTable(metaTableName)) {
- throw new SemanticException(ErrorMsg.INVALID_METADATA_TABLE_NAME, metaTableName);
+ if (tableIdentifier.startsWith("branch_")) {
+ if (!t.getStorageHandler().isValidBranch(tTable, tableIdentifier.substring(7))) {
+ throw new SemanticException(String.format("Cannot use branch (does not exist): %s", tableIdentifier.substring(7)));
+ }
+ t.setBranchName(tableIdentifier);
+ } else {
+ if (!t.getStorageHandler().isValidMetadataTable(tableIdentifier)) {
Review Comment:
Better than `TableIdentifier`. :) I changed `TableIdentifier `to `TableMetaRef`. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1205352972
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -542,14 +542,18 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableIdentifierSupported() {
return false;
}
default boolean isValidMetadataTable(String metaTableName) {
return false;
}
+ default boolean isValidBranch(org.apache.hadoop.hive.metastore.api.Table hmsTable, String branchName) {
Review Comment:
what if we have generic `isValidTableMetaRef ` that could validate metaRef based on supplied type (TableMetaRef{type:enum{branch,tag,metatable}})?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1205363358
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java:
##########
@@ -211,8 +211,13 @@ public boolean isDestToOpTypeInsertOverwrite(String clause) {
/**
* See also {@link #getInsertOverwriteTables()}
*/
- public boolean isInsertIntoTable(String dbName, String table) {
- String fullName = dbName + "." + table;
+ public boolean isInsertIntoTable(String dbName, String table, String branchName) {
+ String fullName;
+ if (branchName != null) {
Review Comment:
````
String fullName = dbName + "." + table;
if (branchName != null) {
fullName += "." + branchName;
}
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1200770165
##########
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java:
##########
@@ -763,6 +763,9 @@ public static TableDesc getTableDesc(Table tbl) {
if (tbl.getMetaTable() != null) {
props.put("metaTable", tbl.getMetaTable());
}
+ if (tbl.getBranchName() != null) {
+ props.put("branch_name", tbl.getBranchName());
Review Comment:
could we move String literal to constant
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1221466338
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -550,14 +550,33 @@ default boolean isTimeTravelAllowed() {
return false;
}
+ /**
+ * Introduced by HIVE-25457 for iceberg to query metadata table.
+ * @return true if the storage handler can support it
+ * @deprecated Use {@link #isTableMetaRefSupported()}
+ */
+ @Deprecated
default boolean isMetadataTableSupported() {
return false;
}
Review Comment:
Thanks. Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn merged pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn merged PR #4292:
URL: https://github.com/apache/hive/pull/4292
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1217832647
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -550,14 +550,19 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableMetaRefSupported() {
return false;
}
Review Comment:
Done. Please take a look again if my doc description is reasonable. Thanks all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: [WIP]HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1544338940
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [11 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1554924992
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [11 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1204003616
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:
##########
@@ -8006,12 +8012,12 @@ && enableColumnStatsCollecting()
&& ColumnStatsAutoGatherContext.canRunAutogatherStats(fso)) {
if (destType == QBMetaData.DEST_TABLE) {
genAutoColumnStatsGatheringPipeline(destinationTable, partSpec, input,
- qb.getParseInfo().isInsertIntoTable(destinationTable.getDbName(), destinationTable.getTableName()),
- false);
+ qb.getParseInfo().isInsertIntoTable(destinationTable.getDbName(), destinationTable.getTableName(),
+ destinationTable.getBranchName()),false);
Review Comment:
Done.
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:
##########
@@ -8006,12 +8012,12 @@ && enableColumnStatsCollecting()
&& ColumnStatsAutoGatherContext.canRunAutogatherStats(fso)) {
if (destType == QBMetaData.DEST_TABLE) {
genAutoColumnStatsGatheringPipeline(destinationTable, partSpec, input,
- qb.getParseInfo().isInsertIntoTable(destinationTable.getDbName(), destinationTable.getTableName()),
- false);
+ qb.getParseInfo().isInsertIntoTable(destinationTable.getDbName(), destinationTable.getTableName(),
+ destinationTable.getBranchName()),false);
} else if (destType == QBMetaData.DEST_PARTITION) {
genAutoColumnStatsGatheringPipeline(destinationTable, destinationPartition.getSpec(), input,
- qb.getParseInfo().isInsertIntoTable(destinationTable.getDbName(), destinationTable.getTableName()),
- false);
+ qb.getParseInfo().isInsertIntoTable(destinationTable.getDbName(), destinationTable.getTableName(),
+ destinationTable.getBranchName()),false);
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1204002311
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -1751,15 +1751,22 @@ public Table getTable(final String dbName, final String tableName, String metaTa
}
Table t = new Table(tTable);
- if (metaTableName != null) {
- if (t.getStorageHandler() == null || !t.getStorageHandler().isMetadataTableSupported()) {
- throw new SemanticException(ErrorMsg.METADATA_TABLE_NOT_SUPPORTED, t.getTableName());
+ if (tableIdentifier != null) {
+ if (t.getStorageHandler() == null || !t.getStorageHandler().isTableIdentifierSupported()) {
+ throw new SemanticException(ErrorMsg.TABLE_IDENTIFIER_NOT_SUPPORTED, t.getTableName());
}
- if (!t.getStorageHandler().isValidMetadataTable(metaTableName)) {
- throw new SemanticException(ErrorMsg.INVALID_METADATA_TABLE_NAME, metaTableName);
+ if (tableIdentifier.startsWith("branch_")) {
Review Comment:
Fixed. I use the added util method `HiveUtils::getTableBranch` to avoid this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1204011092
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java:
##########
@@ -5406,16 +5406,16 @@ protected Table getTableObjectByName(String tabName, boolean throwException) thr
String[] names = Utilities.getDbTableName(tabName);
final String tableName = names[1];
final String dbName = names[0];
- String metaTable = null;
+ String tableIdentifier = null;
Review Comment:
Done. I reused `TableName.fromString` to simplify here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1562122054
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [9 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1205349845
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -1751,15 +1751,22 @@ public Table getTable(final String dbName, final String tableName, String metaTa
}
Table t = new Table(tTable);
- if (metaTableName != null) {
- if (t.getStorageHandler() == null || !t.getStorageHandler().isMetadataTableSupported()) {
- throw new SemanticException(ErrorMsg.METADATA_TABLE_NOT_SUPPORTED, t.getTableName());
+ if (tableIdentifier != null) {
+ if (t.getStorageHandler() == null || !t.getStorageHandler().isTableIdentifierSupported()) {
+ throw new SemanticException(ErrorMsg.TABLE_IDENTIFIER_NOT_SUPPORTED, t.getTableName());
}
- if (!t.getStorageHandler().isValidMetadataTable(metaTableName)) {
- throw new SemanticException(ErrorMsg.INVALID_METADATA_TABLE_NAME, metaTableName);
+ if (tableIdentifier.startsWith("branch_")) {
+ if (!t.getStorageHandler().isValidBranch(tTable, tableIdentifier.substring(7))) {
+ throw new SemanticException(String.format("Cannot use branch (does not exist): %s", tableIdentifier.substring(7)));
+ }
+ t.setBranchName(tableIdentifier);
+ } else {
+ if (!t.getStorageHandler().isValidMetadataTable(tableIdentifier)) {
Review Comment:
I am not really good with naming, but maybe something like `TableIdentifierMetaExtension` / `TableMetaRef`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1205375803
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputCommitter.java:
##########
@@ -451,15 +452,21 @@ private void commitTable(FileIO io, ExecutorService executor, OutputTable output
* @param startTime The start time of the commit - used only for logging
* @param results The object containing the new files we would like to add to the table
*/
- private void commitWrite(Table table, long startTime, FilesForCommit results) {
+ private void commitWrite(Table table, String branchName, long startTime, FilesForCommit results) {
if (results.deleteFiles().isEmpty()) {
AppendFiles write = table.newAppend();
results.dataFiles().forEach(write::appendFile);
+ if (branchName != null) {
+ write.toBranch(branchName.substring(7));
Review Comment:
👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1217641598
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -550,14 +550,19 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableMetaRefSupported() {
return false;
}
Review Comment:
sounds reasonable, however, please add Javadoc for both, and maybe mark isMetadataTableSupported() as deprecated so we could clean the interface in the next releases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1214037388
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -550,14 +550,19 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableMetaRefSupported() {
return false;
}
Review Comment:
> Sure. It is mainly used to check table if having metadata table or branch&tag. Acutually, we think it is only used by iceberg.
it was only introduced for Iceberg to query table metadata, so it should be completely safe to alter the name to a more generic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1213948843
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -550,14 +550,19 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableMetaRefSupported() {
return false;
}
Review Comment:
Sure. It is mainly used to check table if having metadata table or branch&tag. Acutually, we think it is only used by iceberg.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1566116222
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [9 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1577919789
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [10 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: [WIP]HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1546661999
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [11 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: [WIP]HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1546809855
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [11 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1204065743
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -1751,15 +1751,22 @@ public Table getTable(final String dbName, final String tableName, String metaTa
}
Table t = new Table(tTable);
- if (metaTableName != null) {
- if (t.getStorageHandler() == null || !t.getStorageHandler().isMetadataTableSupported()) {
- throw new SemanticException(ErrorMsg.METADATA_TABLE_NOT_SUPPORTED, t.getTableName());
+ if (tableIdentifier != null) {
+ if (t.getStorageHandler() == null || !t.getStorageHandler().isTableIdentifierSupported()) {
+ throw new SemanticException(ErrorMsg.TABLE_IDENTIFIER_NOT_SUPPORTED, t.getTableName());
}
- if (!t.getStorageHandler().isValidMetadataTable(metaTableName)) {
- throw new SemanticException(ErrorMsg.INVALID_METADATA_TABLE_NAME, metaTableName);
+ if (tableIdentifier.startsWith("branch_")) {
+ if (!t.getStorageHandler().isValidBranch(tTable, tableIdentifier.substring(7))) {
+ throw new SemanticException(String.format("Cannot use branch (does not exist): %s", tableIdentifier.substring(7)));
+ }
+ t.setBranchName(tableIdentifier);
+ } else {
+ if (!t.getStorageHandler().isValidMetadataTable(tableIdentifier)) {
Review Comment:
@deniskuzZ Sorry for the confusion. I struggled here for a long time too. Let me explain my understanding.
`isValidMetadataTable ` is only used for iceberg meta table, today we can query iceberg meta table `snapshots `using **three layer namespace** like: ` dbName.tblName.snapshots`
https://github.com/apache/hive/blob/227db3df60152d84994a91dfae789edfd1450bd3/iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/IcebergMetadataTables.java#L22-L26
` isTableIdentifierSupported` I added is a general term which indicate the table can have **three layer namespace** , e.g meta table ` dbName.tblName.snapshots` and branch&tag ` dbName.tblName.branch_branchName`.
That is to say, i want to use `TableIdentifier` to mean the table support **three layer namespace** before we distinguish between meta table and branch using keyword `branch_`.
But maybe the term `TableIdentifier` can not express the meaning of **three layer namespace** precisely. I want to you give some advice. Thanks in advance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1204029620
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -542,14 +542,18 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableIdentifierSupported() {
return false;
}
default boolean isValidMetadataTable(String metaTableName) {
return false;
}
+ default boolean isValidBranch(org.apache.hadoop.hive.metastore.api.Table hmsTable, String branchName) {
Review Comment:
I agree you. But here i want to check the branch validity in `compile ` stage instead of iceberg `commit `stage , and i didn't find good way except using `HiveStorageHandler` to invoke iceberg api in `ql `module.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1204007304
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java:
##########
@@ -211,8 +211,13 @@ public boolean isDestToOpTypeInsertOverwrite(String clause) {
/**
* See also {@link #getInsertOverwriteTables()}
*/
- public boolean isInsertIntoTable(String dbName, String table) {
- String fullName = dbName + "." + table;
+ public boolean isInsertIntoTable(String dbName, String table, String branchName) {
+ String fullName;
+ if (branchName != null) {
Review Comment:
Done. It looks much clean now. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1204009119
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveTableName.java:
##########
@@ -106,6 +106,18 @@ public static TableName ofNullable(String dbTableName, String defaultDb) throws
}
}
+ public static TableName ofNullable(String dbTableName, String defaultDb, String branchName) throws SemanticException {
Review Comment:
Done. Thanks very much for your reminding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1213946754
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java:
##########
@@ -107,6 +109,7 @@ public static String escapeString(String str) {
static final byte[] tabEscapeBytes = "\\t".getBytes();;
static final byte[] tabUnescapeBytes = "\t".getBytes();
static final byte[] ctrlABytes = "\u0001".getBytes();
+ static final Pattern BRANCH = Pattern.compile("branch_(.*)");
Review Comment:
It is only used to extract branch name from the table suffix `tableMetaRef `https://github.com/apache/hive/pull/4292/files#diff-42b2d7dcd39e3f43dd8da948858388f92d2213a4963ab7e08cb92df6cc06453dR40
e.g. the three layer table ref identifier `dbName.tblName.tblMetaRef` --> `testdb.testtbl.branch_test1`, we will get actual branch from the `tblMetaRef` ->`test1`.
So it does't matter if table or database has branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1213966080
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -550,14 +550,19 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableMetaRefSupported() {
return false;
}
Review Comment:
I think we shouldn’t assume, but maintain compat and keep the name as is
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1214037388
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -550,14 +550,19 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableMetaRefSupported() {
return false;
}
Review Comment:
> Sure. It is mainly used to check table if having metadata table or branch&tag. Acutually, we think it is only used by iceberg.
it was only introduced for iceberg, so i don't think it should be an issue to alter the name to a more generic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1200746923
##########
common/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java:
##########
@@ -478,7 +478,7 @@ public enum ErrorMsg {
DATACONNECTOR_NOT_EXISTS(10428, "Dataconnector does not exist:"),
TIME_TRAVEL_NOT_ALLOWED(10429, "Time travel is not allowed for {0}. Please choose a storage format which supports the feature.", true),
INVALID_METADATA_TABLE_NAME(10430, "Invalid metadata table name {0}.", true),
- METADATA_TABLE_NOT_SUPPORTED(10431, "Metadata tables are not supported for table {0}.", true),
+ TABLE_IDENTIFIER_NOT_SUPPORTED(10431, "Table identifier is not supported for table {0}.", true),
Review Comment:
That looks like a different errors, isn't it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1200758637
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputCommitter.java:
##########
@@ -451,15 +452,21 @@ private void commitTable(FileIO io, ExecutorService executor, OutputTable output
* @param startTime The start time of the commit - used only for logging
* @param results The object containing the new files we would like to add to the table
*/
- private void commitWrite(Table table, long startTime, FilesForCommit results) {
+ private void commitWrite(Table table, String branchName, long startTime, FilesForCommit results) {
if (results.deleteFiles().isEmpty()) {
AppendFiles write = table.newAppend();
results.dataFiles().forEach(write::appendFile);
+ if (branchName != null) {
Review Comment:
````
StringUtils.isNotEmpty(branchName)
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1200777000
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -1751,15 +1751,22 @@ public Table getTable(final String dbName, final String tableName, String metaTa
}
Table t = new Table(tTable);
- if (metaTableName != null) {
- if (t.getStorageHandler() == null || !t.getStorageHandler().isMetadataTableSupported()) {
- throw new SemanticException(ErrorMsg.METADATA_TABLE_NOT_SUPPORTED, t.getTableName());
+ if (tableIdentifier != null) {
+ if (t.getStorageHandler() == null || !t.getStorageHandler().isTableIdentifierSupported()) {
+ throw new SemanticException(ErrorMsg.TABLE_IDENTIFIER_NOT_SUPPORTED, t.getTableName());
}
- if (!t.getStorageHandler().isValidMetadataTable(metaTableName)) {
- throw new SemanticException(ErrorMsg.INVALID_METADATA_TABLE_NAME, metaTableName);
+ if (tableIdentifier.startsWith("branch_")) {
Review Comment:
please move string literal "branch_" to constant
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1550620069
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [10 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1566023025
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [9 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1205358745
##########
common/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java:
##########
@@ -478,7 +478,7 @@ public enum ErrorMsg {
DATACONNECTOR_NOT_EXISTS(10428, "Dataconnector does not exist:"),
TIME_TRAVEL_NOT_ALLOWED(10429, "Time travel is not allowed for {0}. Please choose a storage format which supports the feature.", true),
INVALID_METADATA_TABLE_NAME(10430, "Invalid metadata table name {0}.", true),
- METADATA_TABLE_NOT_SUPPORTED(10431, "Metadata tables are not supported for table {0}.", true),
+ TABLE_IDENTIFIER_NOT_SUPPORTED(10431, "Table identifier is not supported for table {0}.", true),
Review Comment:
maybe `Table Metadata Ref extension not supported for table`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1205364525
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java:
##########
@@ -5406,16 +5406,16 @@ protected Table getTableObjectByName(String tabName, boolean throwException) thr
String[] names = Utilities.getDbTableName(tabName);
final String tableName = names[1];
final String dbName = names[0];
- String metaTable = null;
+ String tableIdentifier = null;
if (names.length == 3) {
- metaTable = names[2];
+ tableIdentifier = names[2];
}
String fullyQualName = dbName + "." + tableName;
- if (metaTable != null) {
- fullyQualName = fullyQualName + "." + metaTable;
+ if (tableIdentifier != null) {
+ fullyQualName = fullyQualName + "." + tableIdentifier;
Review Comment:
````
fullyQualName += "." + tableIdentifier;
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1208395291
##########
common/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java:
##########
@@ -478,7 +478,7 @@ public enum ErrorMsg {
DATACONNECTOR_NOT_EXISTS(10428, "Dataconnector does not exist:"),
TIME_TRAVEL_NOT_ALLOWED(10429, "Time travel is not allowed for {0}. Please choose a storage format which supports the feature.", true),
INVALID_METADATA_TABLE_NAME(10430, "Invalid metadata table name {0}.", true),
- METADATA_TABLE_NOT_SUPPORTED(10431, "Metadata tables are not supported for table {0}.", true),
+ TABLE_IDENTIFIER_NOT_SUPPORTED(10431, "Table identifier is not supported for table {0}.", true),
Review Comment:
I like this. It's better than identifier. Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1203984663
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergInputFormat.java:
##########
@@ -143,6 +144,8 @@ public InputSplit[] getSplits(JobConf job, int numSplits) throws IOException {
job.set(InputFormatConfig.AS_OF_TIMESTAMP, job.get(TableScanDesc.AS_OF_TIMESTAMP, "-1"));
job.set(InputFormatConfig.SNAPSHOT_ID, job.get(TableScanDesc.AS_OF_VERSION, "-1"));
job.set(InputFormatConfig.SNAPSHOT_ID_INTERVAL_FROM, job.get(TableScanDesc.FROM_VERSION, "-1"));
+ Optional.ofNullable(job.get(TableScanDesc.BRANCH_NAME)).ifPresent(branchName ->
Review Comment:
Fixed. Thanks.
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputCommitter.java:
##########
@@ -451,15 +452,21 @@ private void commitTable(FileIO io, ExecutorService executor, OutputTable output
* @param startTime The start time of the commit - used only for logging
* @param results The object containing the new files we would like to add to the table
*/
- private void commitWrite(Table table, long startTime, FilesForCommit results) {
+ private void commitWrite(Table table, String branchName, long startTime, FilesForCommit results) {
if (results.deleteFiles().isEmpty()) {
AppendFiles write = table.newAppend();
results.dataFiles().forEach(write::appendFile);
+ if (branchName != null) {
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1203984332
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/InputFormatConfig.java:
##########
@@ -85,6 +85,7 @@ private InputFormatConfig() {
public static final boolean CONFIG_SERIALIZATION_DISABLED_DEFAULT = true;
public static final String OPERATION_TYPE_PREFIX = "iceberg.mr.operation.type.";
public static final String OUTPUT_TABLES = "iceberg.mr.output.tables";
+ public static final String OUTPUT_TABLE_BRANCH = "iceberg.mr.output.table_branch";
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1200762039
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##########
@@ -276,6 +276,10 @@ public void configureJobConf(TableDesc tableDesc, JobConf jobConf) {
String tables = jobConf.get(InputFormatConfig.OUTPUT_TABLES);
tables = tables == null ? tableName : tables + TABLE_NAME_SEPARATOR + tableName;
jobConf.set(InputFormatConfig.OUTPUT_TABLES, tables);
+ // todo remove
Review Comment:
is this leftover?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1200752594
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/InputFormatConfig.java:
##########
@@ -85,6 +85,7 @@ private InputFormatConfig() {
public static final boolean CONFIG_SERIALIZATION_DISABLED_DEFAULT = true;
public static final String OPERATION_TYPE_PREFIX = "iceberg.mr.operation.type.";
public static final String OUTPUT_TABLES = "iceberg.mr.output.tables";
+ public static final String OUTPUT_TABLE_BRANCH = "iceberg.mr.output.table_branch";
Review Comment:
should we use dot "." separator to be consistent with other properties?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1201940330
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java:
##########
@@ -5406,16 +5406,16 @@ protected Table getTableObjectByName(String tabName, boolean throwException) thr
String[] names = Utilities.getDbTableName(tabName);
final String tableName = names[1];
final String dbName = names[0];
- String metaTable = null;
+ String tableIdentifier = null;
Review Comment:
The below logic looks similar to `TableName.fromString`, could we reuse/optimize here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1201936910
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveTableName.java:
##########
@@ -106,6 +106,18 @@ public static TableName ofNullable(String dbTableName, String defaultDb) throws
}
}
+ public static TableName ofNullable(String dbTableName, String defaultDb, String branchName) throws SemanticException {
Review Comment:
why don't you reuse new signature in `ofNullable(String dbTableName, String defaultDb)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1214000613
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -550,14 +550,19 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableMetaRefSupported() {
return false;
}
Review Comment:
ha, i am not obect to keep the method as it is. Let me give some thought.
The `isMetadataTableSupported()` is only used here https://github.com/apache/hive/pull/4292/files#diff-9d6bc2f1a2972d844b3107f29bc1731af7adf6f4c922e08904345c5ae88cd727L1755, and then `storagehandler::isValidMetadataTable` will check the metadata table if is valid.
That is to say, the actual effect of method `isMetadataTableSupported()` is just simply to verify that the table has three layer identifier `database.table.tableRef`, and the final check occurs in `storagehandler::isValidMetadataTable`. And i rename it from `isMetadataTableSupported()` to `isTableMetaRefSupported()` to make its three_layer_identifier meaning more general, not just for metadata table.
But maybe i am not good at naming & renaming method. If you think we should keep hte name as is or using another method name, please feel free to give advice. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: [WIP]HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1537128993
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [11 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1576632014
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [9 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1201920602
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:
##########
@@ -8006,12 +8012,12 @@ && enableColumnStatsCollecting()
&& ColumnStatsAutoGatherContext.canRunAutogatherStats(fso)) {
if (destType == QBMetaData.DEST_TABLE) {
genAutoColumnStatsGatheringPipeline(destinationTable, partSpec, input,
- qb.getParseInfo().isInsertIntoTable(destinationTable.getDbName(), destinationTable.getTableName()),
- false);
+ qb.getParseInfo().isInsertIntoTable(destinationTable.getDbName(), destinationTable.getTableName(),
+ destinationTable.getBranchName()),false);
Review Comment:
nit: missing space
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1205364742
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java:
##########
@@ -5406,16 +5406,16 @@ protected Table getTableObjectByName(String tabName, boolean throwException) thr
String[] names = Utilities.getDbTableName(tabName);
final String tableName = names[1];
final String dbName = names[0];
- String metaTable = null;
+ String tableIdentifier = null;
Review Comment:
that's fine
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1205349845
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -1751,15 +1751,22 @@ public Table getTable(final String dbName, final String tableName, String metaTa
}
Table t = new Table(tTable);
- if (metaTableName != null) {
- if (t.getStorageHandler() == null || !t.getStorageHandler().isMetadataTableSupported()) {
- throw new SemanticException(ErrorMsg.METADATA_TABLE_NOT_SUPPORTED, t.getTableName());
+ if (tableIdentifier != null) {
+ if (t.getStorageHandler() == null || !t.getStorageHandler().isTableIdentifierSupported()) {
+ throw new SemanticException(ErrorMsg.TABLE_IDENTIFIER_NOT_SUPPORTED, t.getTableName());
}
- if (!t.getStorageHandler().isValidMetadataTable(metaTableName)) {
- throw new SemanticException(ErrorMsg.INVALID_METADATA_TABLE_NAME, metaTableName);
+ if (tableIdentifier.startsWith("branch_")) {
+ if (!t.getStorageHandler().isValidBranch(tTable, tableIdentifier.substring(7))) {
+ throw new SemanticException(String.format("Cannot use branch (does not exist): %s", tableIdentifier.substring(7)));
+ }
+ t.setBranchName(tableIdentifier);
+ } else {
+ if (!t.getStorageHandler().isValidMetadataTable(tableIdentifier)) {
Review Comment:
I am not really good with naming, but maybe something like `TableMetaRef`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1203995226
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##########
@@ -276,6 +276,10 @@ public void configureJobConf(TableDesc tableDesc, JobConf jobConf) {
String tables = jobConf.get(InputFormatConfig.OUTPUT_TABLES);
tables = tables == null ? tableName : tables + TABLE_NAME_SEPARATOR + tableName;
jobConf.set(InputFormatConfig.OUTPUT_TABLES, tables);
+ // todo remove
Review Comment:
Thanks. It was added when debugging code.
Removed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1208405219
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -542,14 +542,18 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableIdentifierSupported() {
return false;
}
default boolean isValidMetadataTable(String metaTableName) {
return false;
}
+ default boolean isValidBranch(org.apache.hadoop.hive.metastore.api.Table hmsTable, String branchName) {
Review Comment:
I removed this method `isValidBranch` and added a new method `HiveStorageHandler::checkAndSetTableMetaRef` which can check and set tableMetaRef(metadata, branch, tag), and this can make `Hive::getTable` of `ql ` module more clean as the tableMetaRef is specific to iceberg.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1208395774
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##########
@@ -661,7 +666,7 @@ public boolean isTimeTravelAllowed() {
}
@Override
- public boolean isMetadataTableSupported() {
+ public boolean isTableIdentifierSupported() {
Review Comment:
Changed this method to `isTableMetaRefSupported()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1214022941
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -550,14 +550,19 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableMetaRefSupported() {
return false;
}
Review Comment:
Should keep the name as is IMO
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1213948466
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java:
##########
@@ -107,6 +109,7 @@ public static String escapeString(String str) {
static final byte[] tabEscapeBytes = "\\t".getBytes();;
static final byte[] tabUnescapeBytes = "\t".getBytes();
static final byte[] ctrlABytes = "\u0001".getBytes();
+ static final Pattern BRANCH = Pattern.compile("branch_(.*)");
Review Comment:
Cool
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1215132378
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -550,14 +550,19 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableMetaRefSupported() {
return false;
}
Review Comment:
We don't just have it in Iceberg code:
https://github.com/apache/hive/blob/rel/release-4.0.0-alpha-2/ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java#L464-L465
We added in the parent interface itself and it is indeed released in the previous release also.
https://github.com/apache/hive/blob/rel/release-4.0.0-alpha-2/ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java#L464-L465
So, technically if some client code does getStorageHandler() and calls this it is gonna fail, or if someone had a custom/internal storage handler where he implemented it, he is gonna suffer as well.
Mostly nobody would have, but we can't gurantee based on your sentiments :-), But it is still an incompatible change. "Changing a public method name/signature", I think this method name change ain't serving any functional benefits, we can keep it as is
If people still want it: The correct way is deprecate the original, add a new one, call the new one from the deprecated method. But I don't think we want it that hard. I would be against breaking compat just for look & feel :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1581000559
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [19 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1577357335
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [10 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: [WIP]HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1537182687
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [11 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1200777832
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -1751,15 +1751,22 @@ public Table getTable(final String dbName, final String tableName, String metaTa
}
Table t = new Table(tTable);
- if (metaTableName != null) {
- if (t.getStorageHandler() == null || !t.getStorageHandler().isMetadataTableSupported()) {
- throw new SemanticException(ErrorMsg.METADATA_TABLE_NOT_SUPPORTED, t.getTableName());
+ if (tableIdentifier != null) {
+ if (t.getStorageHandler() == null || !t.getStorageHandler().isTableIdentifierSupported()) {
+ throw new SemanticException(ErrorMsg.TABLE_IDENTIFIER_NOT_SUPPORTED, t.getTableName());
}
- if (!t.getStorageHandler().isValidMetadataTable(metaTableName)) {
- throw new SemanticException(ErrorMsg.INVALID_METADATA_TABLE_NAME, metaTableName);
+ if (tableIdentifier.startsWith("branch_")) {
+ if (!t.getStorageHandler().isValidBranch(tTable, tableIdentifier.substring(7))) {
+ throw new SemanticException(String.format("Cannot use branch (does not exist): %s", tableIdentifier.substring(7)));
Review Comment:
move to constant
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1201930835
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java:
##########
@@ -211,8 +211,13 @@ public boolean isDestToOpTypeInsertOverwrite(String clause) {
/**
* See also {@link #getInsertOverwriteTables()}
*/
- public boolean isInsertIntoTable(String dbName, String table) {
- String fullName = dbName + "." + table;
+ public boolean isInsertIntoTable(String dbName, String table, String branchName) {
+ String fullName;
+ if (branchName != null) {
Review Comment:
could we have some helper for that to avoid duplication in multiple places:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1203992759
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputCommitter.java:
##########
@@ -451,15 +452,21 @@ private void commitTable(FileIO io, ExecutorService executor, OutputTable output
* @param startTime The start time of the commit - used only for logging
* @param results The object containing the new files we would like to add to the table
*/
- private void commitWrite(Table table, long startTime, FilesForCommit results) {
+ private void commitWrite(Table table, String branchName, long startTime, FilesForCommit results) {
if (results.deleteFiles().isEmpty()) {
AppendFiles write = table.newAppend();
results.dataFiles().forEach(write::appendFile);
+ if (branchName != null) {
+ write.toBranch(branchName.substring(7));
Review Comment:
I added a new util method `HiveUtils::getTableBranch` in `ql `module to extract the table's branch, so that `ql` module an `iceberg `handler module can use this method to check and get branch info.
https://github.com/apache/hive/pull/4292/files#diff-9a5333458ddbc266f12512f60bd04fddd9deae5a76648eb109dbabdc0e13fc30R446
Please check if it is ok. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1200759844
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputCommitter.java:
##########
@@ -451,15 +452,21 @@ private void commitTable(FileIO io, ExecutorService executor, OutputTable output
* @param startTime The start time of the commit - used only for logging
* @param results The object containing the new files we would like to add to the table
*/
- private void commitWrite(Table table, long startTime, FilesForCommit results) {
+ private void commitWrite(Table table, String branchName, long startTime, FilesForCommit results) {
if (results.deleteFiles().isEmpty()) {
AppendFiles write = table.newAppend();
results.dataFiles().forEach(write::appendFile);
+ if (branchName != null) {
+ write.toBranch(branchName.substring(7));
Review Comment:
magic number - 7
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1204071713
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##########
@@ -661,7 +666,7 @@ public boolean isTimeTravelAllowed() {
}
@Override
- public boolean isMetadataTableSupported() {
+ public boolean isTableIdentifierSupported() {
Review Comment:
I want to use a general term **TableIdentifier** which indicates the table can have `three layer namespace` , e.g meta table `dbName.tblName.snapshots` and branch&tag `dbName.tblName.branch_branchName.`
I explained this in details in following comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1561758049
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [9 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1565969599
> LGTM, thanks @zhangbutao for the patch! I've left a few minor comments.
Thank you very much for your comments! @deniskuzZ
I have made new fix. Please take a look again if you have a chance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zhangbutao commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1208405617
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java:
##########
@@ -211,8 +211,13 @@ public boolean isDestToOpTypeInsertOverwrite(String clause) {
/**
* See also {@link #getInsertOverwriteTables()}
*/
- public boolean isInsertIntoTable(String dbName, String table) {
- String fullName = dbName + "." + table;
+ public boolean isInsertIntoTable(String dbName, String table, String branchName) {
+ String fullName;
+ if (branchName != null) {
Review Comment:
Done. Thanks.
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java:
##########
@@ -5406,16 +5406,16 @@ protected Table getTableObjectByName(String tabName, boolean throwException) thr
String[] names = Utilities.getDbTableName(tabName);
final String tableName = names[1];
final String dbName = names[0];
- String metaTable = null;
+ String tableIdentifier = null;
if (names.length == 3) {
- metaTable = names[2];
+ tableIdentifier = names[2];
}
String fullyQualName = dbName + "." + tableName;
- if (metaTable != null) {
- fullyQualName = fullyQualName + "." + metaTable;
+ if (tableIdentifier != null) {
+ fullyQualName = fullyQualName + "." + tableIdentifier;
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1205352972
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -542,14 +542,18 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableIdentifierSupported() {
return false;
}
default boolean isValidMetadataTable(String metaTableName) {
return false;
}
+ default boolean isValidBranch(org.apache.hadoop.hive.metastore.api.Table hmsTable, String branchName) {
Review Comment:
what if we have generic `isValidTableMetaRef ` that could validate metaRef based on supplied type: TableMetaRef{type: enum(branch, tag, metatable)}?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: [WIP]HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1535526849
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: [WIP]HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1537033307
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [13 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4292:
URL: https://github.com/apache/hive/pull/4292#discussion_r1214037388
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java:
##########
@@ -550,14 +550,19 @@ default boolean isTimeTravelAllowed() {
return false;
}
- default boolean isMetadataTableSupported() {
+ default boolean isTableMetaRefSupported() {
return false;
}
Review Comment:
> Sure. It is mainly used to check table if having metadata table or branch&tag. Acutually, we think it is only used by iceberg.
it was only introduced for iceberg, so i don't think it should be an issue to alter the name
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1581916978
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [10 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4292: HIVE-27302: Iceberg: Suport write to iceberg branch
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4292:
URL: https://github.com/apache/hive/pull/4292#issuecomment-1578135316
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4292)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4292&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL) [10 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4292&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4292&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org