You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/10/05 21:50:00 UTC
[jira] [Work logged] (HIVE-19253) HMS ignores tableType property
for external tables
[ https://issues.apache.org/jira/browse/HIVE-19253?focusedWorklogId=495600&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495600 ]
ASF GitHub Bot logged work on HIVE-19253:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 05/Oct/20 21:49
Start Date: 05/Oct/20 21:49
Worklog Time Spent: 10m
Work Description: vihangk1 commented on a change in pull request #1537:
URL: https://github.com/apache/hive/pull/1537#discussion_r499890809
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestObjectStore.java
##########
@@ -931,6 +924,64 @@ public void testNotificationOps() throws InterruptedException, MetaException {
Assert.assertEquals(0, eventResponse.getEventsSize());
}
+ /**
+ * Verify that table type is set correctly based on input table properties.
+ * Two things are verified:
+ * <ol>
+ * <li>When <em>EXTERNAL</em> property is set to <em>true</em>, table type should be external</li>
+ * <li>When table type is set to external it should remain external</em>
+ * </ol>
+ * @throws Exception
+ */
+ @Test
+ public void testExternalTable() throws Exception {
+ Database db1 = new DatabaseBuilder()
+ .setName(DB1)
+ .setDescription("description")
+ .setLocation("locationurl")
+ .build(conf);
+ objectStore.createDatabase(db1);
+
+ List<Table> tables = new ArrayList<>(4);
+ Map<String, Boolean> expectedValues = new HashMap<>();
+
+ int i = 1;
+ // Case 1: EXTERNAL = true, tableType == MANAGED_TABLE
+ // The result should be external table
+ Table tbl1 = buildTable(conf, db1, "t" + i++, true, null);
+ tables.add(tbl1);
+ expectedValues.put(tbl1.getTableName(), true);
+ // Case 2: EXTERNAL = false, tableType == EXTERNAL_TABLE
+ // The result should be external table
+ Table tbl2 = buildTable(conf, db1, "t" + i++, false, TableType.EXTERNAL_TABLE.name());
+ tables.add(tbl2);
+ expectedValues.put(tbl2.getTableName(), true);
+ // Case 3: EXTERNAL = false, tableType == EXTERNAL_TABLE
Review comment:
the comment should state EXTERNAL = true
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestObjectStore.java
##########
@@ -84,15 +85,7 @@
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.HashMap;
-import java.util.HashSet;
-import java.util.LinkedList;
-import java.util.List;
-import java.util.Random;
-import java.util.Set;
-import java.util.UUID;
+import java.util.*;
Review comment:
this change can be reverted since we don't use wildcard imports.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 495600)
Time Spent: 20m (was: 10m)
> HMS ignores tableType property for external tables
> --------------------------------------------------
>
> Key: HIVE-19253
> URL: https://issues.apache.org/jira/browse/HIVE-19253
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 3.0.0, 3.1.0, 4.0.0
> Reporter: Alex Kolbasov
> Assignee: Vihang Karajgaonkar
> Priority: Major
> Labels: newbie, pull-request-available
> Attachments: HIVE-19253.01.patch, HIVE-19253.02.patch, HIVE-19253.03.patch, HIVE-19253.03.patch, HIVE-19253.04.patch, HIVE-19253.05.patch, HIVE-19253.06.patch, HIVE-19253.07.patch, HIVE-19253.08.patch, HIVE-19253.09.patch, HIVE-19253.10.patch, HIVE-19253.11.patch, HIVE-19253.12.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When someone creates a table using Thrift API they may think that setting tableType to {{EXTERNAL_TABLE}} creates an external table. And boom - their table is gone later because HMS will silently change it to managed table.
> here is the offending code:
> {code:java}
> private MTable convertToMTable(Table tbl) throws InvalidObjectException,
> MetaException {
> ...
> // If the table has property EXTERNAL set, update table type
> // accordingly
> String tableType = tbl.getTableType();
> boolean isExternal = Boolean.parseBoolean(tbl.getParameters().get("EXTERNAL"));
> if (TableType.MANAGED_TABLE.toString().equals(tableType)) {
> if (isExternal) {
> tableType = TableType.EXTERNAL_TABLE.toString();
> }
> }
> if (TableType.EXTERNAL_TABLE.toString().equals(tableType)) {
> if (!isExternal) { // Here!
> tableType = TableType.MANAGED_TABLE.toString();
> }
> }
> {code}
> So if the EXTERNAL parameter is not set, table type is changed to managed even if it was external in the first place - which is wrong.
> More over, in other places code looks at the table property to decide table type and some places look at parameter. HMS should really make its mind which one to use.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)