You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Hive QA (JIRA)" <ji...@apache.org> on 2019/01/16 04:24:00 UTC
[jira] [Commented] (HIVE-19253) HMS ignores tableType property for
external tables
[ https://issues.apache.org/jira/browse/HIVE-19253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743602#comment-16743602 ]
Hive QA commented on HIVE-19253:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943121/HIVE-19253.12.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/15634/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15634/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15634/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-01-16 04:22:42.559
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-15634/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-01-16 04:22:42.562
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at a3aa074 HIVE-21113: For HPL/SQL that contains boolean expression with NOT, incorrect SQL may be generated (Baoning He, reviewed by Daniel Dai)
+ git clean -f -d
Removing ${project.basedir}/
Removing itests/${project.basedir}/
Removing standalone-metastore/metastore-client/
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at a3aa074 HIVE-21113: For HPL/SQL that contains boolean expression with NOT, incorrect SQL may be generated (Baoning He, reviewed by Daniel Dai)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-01-16 04:22:43.591
+ rm -rf ../yetus_PreCommit-HIVE-Build-15634
+ mkdir ../yetus_PreCommit-HIVE-Build-15634
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-15634
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-15634/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch
error: a/hcatalog/webhcat/java-client/src/main/java/org/apache/hive/hcatalog/api/HCatTable.java: does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/util/UpgradeTool.java: does not exist in index
error: a/standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java: does not exist in index
error: a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java: does not exist in index
error: a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java: does not exist in index
error: a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java: does not exist in index
error: a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/TransactionalValidationListener.java: does not exist in index
error: a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java: does not exist in index
error: a/standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestObjectStore.java: does not exist in index
error: patch failed: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestObjectStore.java:86
Falling back to three-way merge...
Applied patch to 'standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestObjectStore.java' with conflicts.
Going to apply patch with: git apply -p1
error: patch failed: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestObjectStore.java:86
Falling back to three-way merge...
Applied patch to 'standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestObjectStore.java' with conflicts.
U standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestObjectStore.java
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-15634
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943121 - PreCommit-HIVE-Build
> HMS ignores tableType property for external tables
> --------------------------------------------------
>
> Key: HIVE-19253
> URL: https://issues.apache.org/jira/browse/HIVE-19253
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 3.0.0, 3.1.0, 4.0.0
> Reporter: Alexander Kolbasov
> Assignee: Vihang Karajgaonkar
> Priority: Major
> Labels: newbie
> Attachments: HIVE-19253.01.patch, HIVE-19253.02.patch, HIVE-19253.03.patch, HIVE-19253.03.patch, HIVE-19253.04.patch, HIVE-19253.05.patch, HIVE-19253.06.patch, HIVE-19253.07.patch, HIVE-19253.08.patch, HIVE-19253.09.patch, HIVE-19253.10.patch, HIVE-19253.11.patch, HIVE-19253.12.patch
>
>
> When someone creates a table using Thrift API they may think that setting tableType to {{EXTERNAL_TABLE}} creates an external table. And boom - their table is gone later because HMS will silently change it to managed table.
> here is the offending code:
> {code:java}
> private MTable convertToMTable(Table tbl) throws InvalidObjectException,
> MetaException {
> ...
> // If the table has property EXTERNAL set, update table type
> // accordingly
> String tableType = tbl.getTableType();
> boolean isExternal = Boolean.parseBoolean(tbl.getParameters().get("EXTERNAL"));
> if (TableType.MANAGED_TABLE.toString().equals(tableType)) {
> if (isExternal) {
> tableType = TableType.EXTERNAL_TABLE.toString();
> }
> }
> if (TableType.EXTERNAL_TABLE.toString().equals(tableType)) {
> if (!isExternal) { // Here!
> tableType = TableType.MANAGED_TABLE.toString();
> }
> }
> {code}
> So if the EXTERNAL parameter is not set, table type is changed to managed even if it was external in the first place - which is wrong.
> More over, in other places code looks at the table property to decide table type and some places look at parameter. HMS should really make its mind which one to use.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)