You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2020/10/14 05:48:11 UTC
[GitHub] [hive] nrg4878 opened a new pull request #1578: HIVE-24271: Managed keyword in create table should always create ACID…
nrg4878 opened a new pull request #1578:
URL: https://github.com/apache/hive/pull/1578
… table (Naveen Gangam)
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/Hive/HowToContribute
2. Ensure that you have created an issue on the Hive project JIRA: https://issues.apache.org/jira/projects/HIVE/summary
3. Ensure you have added or run the appropriate tests for your PR:
4. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP]HIVE-XXXXX: Your PR title ...'.
5. Be sure to keep the PR description updated to reflect all changes.
6. Please write your PR title to summarize what this PR proposes.
7. If possible, provide a concise example to reproduce the issue for a faster review.
-->
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
### Why are the changes needed?
Currently "create managed table" does not create a ACID table when hive.create.as.acid=false and hive.create.as.insert.only=false on the HS2 server. This DDL should create an ACID table regardless of the server settings.
### Does this PR introduce _any_ user-facing change?
NO
### How was this patch tested?
Manually tested the fix.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] nrg4878 commented on a change in pull request #1578: HIVE-24271: Managed keyword in create table should always create ACID…
Posted by GitBox <gi...@apache.org>.
nrg4878 commented on a change in pull request #1578:
URL: https://github.com/apache/hive/pull/1578#discussion_r505958125
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
##########
@@ -13194,16 +13194,16 @@ private void updateDefaultTblProps(Map<String, String> source, Map<String, Strin
}
}
- boolean makeInsertOnly = !isTemporaryTable && (isManaged && HiveConf.getBoolVar(
- conf, ConfVars.HIVE_CREATE_TABLES_AS_INSERT_ONLY));
- boolean makeAcid = !isTemporaryTable && !makeInsertOnly && makeAcid();
+ boolean makeInsertOnly = !isTemporaryTable && HiveConf.getBoolVar(
+ conf, ConfVars.HIVE_CREATE_TABLES_AS_INSERT_ONLY);
+ boolean makeAcid = !isTemporaryTable && makeAcid();
Review comment:
correct .. no change here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] nrg4878 commented on pull request #1578: HIVE-24271: Managed keyword in create table should always create ACID…
Posted by GitBox <gi...@apache.org>.
nrg4878 commented on pull request #1578:
URL: https://github.com/apache/hive/pull/1578#issuecomment-709669794
Thanks for the review @yongzhi Fix has been committed to master.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] yongzhi commented on a change in pull request #1578: HIVE-24271: Managed keyword in create table should always create ACID…
Posted by GitBox <gi...@apache.org>.
yongzhi commented on a change in pull request #1578:
URL: https://github.com/apache/hive/pull/1578#discussion_r505587450
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
##########
@@ -13212,7 +13212,7 @@ private void updateDefaultTblProps(Map<String, String> source, Map<String, Strin
retValue.put(hive_metastoreConstants.TABLE_TRANSACTIONAL_PROPERTIES,
TransactionalValidationListener.INSERTONLY_TRANSACTIONAL_PROPERTY);
}
- if (makeAcid || isTransactional) {
+ if (makeAcid || isTransactional || (isManaged && !makeInsertOnly)) {
Review comment:
Do we have the case it is isTransaction = true, isManaged = true and makeInsertOnly is false (happen not to set TABLE_TRANSACTIONAL_PROPERTIES.) ?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
##########
@@ -13194,16 +13194,16 @@ private void updateDefaultTblProps(Map<String, String> source, Map<String, Strin
}
}
- boolean makeInsertOnly = !isTemporaryTable && (isManaged && HiveConf.getBoolVar(
- conf, ConfVars.HIVE_CREATE_TABLES_AS_INSERT_ONLY));
- boolean makeAcid = !isTemporaryTable && !makeInsertOnly && makeAcid();
+ boolean makeInsertOnly = !isTemporaryTable && HiveConf.getBoolVar(
+ conf, ConfVars.HIVE_CREATE_TABLES_AS_INSERT_ONLY);
+ boolean makeAcid = !isTemporaryTable && makeAcid();
// if not specify managed table and create.table.as.external is true
// ignore makeInsertOnly and makeAcid.
if (!isManaged && HiveConf.getBoolVar(conf, ConfVars.CREATE_TABLE_AS_EXTERNAL)) {
makeInsertOnly = false;
makeAcid = false;
}
- if ((makeInsertOnly || makeAcid || isTransactional)
+ if ((makeInsertOnly || makeAcid || isTransactional || isManaged)
&& !isExt && !isMaterialization && StringUtils.isBlank(storageFormat.getStorageHandler())
Review comment:
I do not understand the logic here: Why isManaged and without explicitly saying it is a transactional table, it is an insert-only one?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] nrg4878 commented on a change in pull request #1578: HIVE-24271: Managed keyword in create table should always create ACID…
Posted by GitBox <gi...@apache.org>.
nrg4878 commented on a change in pull request #1578:
URL: https://github.com/apache/hive/pull/1578#discussion_r505724698
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
##########
@@ -13194,16 +13194,16 @@ private void updateDefaultTblProps(Map<String, String> source, Map<String, Strin
}
}
- boolean makeInsertOnly = !isTemporaryTable && (isManaged && HiveConf.getBoolVar(
- conf, ConfVars.HIVE_CREATE_TABLES_AS_INSERT_ONLY));
- boolean makeAcid = !isTemporaryTable && !makeInsertOnly && makeAcid();
+ boolean makeInsertOnly = !isTemporaryTable && HiveConf.getBoolVar(
+ conf, ConfVars.HIVE_CREATE_TABLES_AS_INSERT_ONLY);
+ boolean makeAcid = !isTemporaryTable && makeAcid();
// if not specify managed table and create.table.as.external is true
// ignore makeInsertOnly and makeAcid.
if (!isManaged && HiveConf.getBoolVar(conf, ConfVars.CREATE_TABLE_AS_EXTERNAL)) {
makeInsertOnly = false;
makeAcid = false;
}
- if ((makeInsertOnly || makeAcid || isTransactional)
+ if ((makeInsertOnly || makeAcid || isTransactional || isManaged)
&& !isExt && !isMaterialization && StringUtils.isBlank(storageFormat.getStorageHandler())
Review comment:
multiple ways to create acid table
create transactional table
create managed table
create table with hive.create.as.acid=true or hive.create.as.insert.only=true.
so the above logic ensures that table is ACID table when managed keyword if used.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] nrg4878 commented on a change in pull request #1578: HIVE-24271: Managed keyword in create table should always create ACID…
Posted by GitBox <gi...@apache.org>.
nrg4878 commented on a change in pull request #1578:
URL: https://github.com/apache/hive/pull/1578#discussion_r505727119
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
##########
@@ -13212,7 +13212,7 @@ private void updateDefaultTblProps(Map<String, String> source, Map<String, Strin
retValue.put(hive_metastoreConstants.TABLE_TRANSACTIONAL_PROPERTIES,
TransactionalValidationListener.INSERTONLY_TRANSACTIONAL_PROPERTY);
}
- if (makeAcid || isTransactional) {
+ if (makeAcid || isTransactional || (isManaged && !makeInsertOnly)) {
Review comment:
so isManaged and isTransactional cannot be true at the time.
create managed table ... -->isManaged=true
create transactional table -->isTransactional=true
with either of the above set, makeInsertOnly can be true or false. This is the server side setting not driven by the query syntax.
so create managed creates "insert_only" acid table if "hive.create.as.insert.only"=true otherwise it creates a full acid table
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] nrg4878 closed pull request #1578: HIVE-24271: Managed keyword in create table should always create ACID…
Posted by GitBox <gi...@apache.org>.
nrg4878 closed pull request #1578:
URL: https://github.com/apache/hive/pull/1578
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] yongzhi commented on a change in pull request #1578: HIVE-24271: Managed keyword in create table should always create ACID…
Posted by GitBox <gi...@apache.org>.
yongzhi commented on a change in pull request #1578:
URL: https://github.com/apache/hive/pull/1578#discussion_r505580388
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
##########
@@ -13194,16 +13194,16 @@ private void updateDefaultTblProps(Map<String, String> source, Map<String, Strin
}
}
- boolean makeInsertOnly = !isTemporaryTable && (isManaged && HiveConf.getBoolVar(
- conf, ConfVars.HIVE_CREATE_TABLES_AS_INSERT_ONLY));
- boolean makeAcid = !isTemporaryTable && !makeInsertOnly && makeAcid();
+ boolean makeInsertOnly = !isTemporaryTable && HiveConf.getBoolVar(
+ conf, ConfVars.HIVE_CREATE_TABLES_AS_INSERT_ONLY);
+ boolean makeAcid = !isTemporaryTable && makeAcid();
Review comment:
Combined the two commits, this part no change, right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org