You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Vihang Karajgaonkar (Jira)" <ji...@apache.org> on 2020/01/29 21:18:00 UTC
[jira] [Updated] (IMPALA-9092) Add support for creating external
Kudu table
[ https://issues.apache.org/jira/browse/IMPALA-9092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vihang Karajgaonkar updated IMPALA-9092:
----------------------------------------
Summary: Add support for creating external Kudu table (was: Fix "show create table" tests on USE_CDP_HIVE=true to account for HIVE-22158)
> Add support for creating external Kudu table
> --------------------------------------------
>
> Key: IMPALA-9092
> URL: https://issues.apache.org/jira/browse/IMPALA-9092
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.4.0
> Reporter: Joe McDonnell
> Assignee: Vihang Karajgaonkar
> Priority: Blocker
> Fix For: Impala 3.4.0
>
>
> Hive changed behavior with HIVE-22158 so that only transactional tables are considered managed and all other are considered external. This means that a regular "create table" will result in an external table with table properties of 'TRANSLATED_TO_EXTERNAL'='TRUE', 'external.table.purge'='TRUE'. This breaks our tests that rely on "show create table", because the table is newly external and has extra table properties. For example:
> {noformat}
> query_test/test_kudu.py:842: in test_primary_key_and_distribution
> db=cursor.conn.db_name, kudu_addr=KUDU_MASTER_HOSTS))
> query_test/test_kudu.py:824: in assert_show_create_equals
> assert cursor.fetchall()[0][0] == \
> E assert "CREATE EXTER...='localhost')" == "CREATE TABLE ...='localhost')"
> E - CREATE EXTERNAL TABLE testshowcreatetable_15312_ggn1hk.nvbpxfuxze
> E ? ---------
> E + CREATE TABLE testshowcreatetable_15312_ggn1hk.nvbpxfuxze (
> E ? ++
> E + c INT NOT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
> E + PRIMARY KEY (c)
> E + )
> E + PARTITION BY HASH (c) PARTITIONS 3
> E STORED AS KUDU
> E - TBLPROPERTIES ('TRANSLATED_TO_EXTERNAL'='TRUE', 'external.table.purge'='TRUE', 'kudu.master_addresses'='localhost')
> E + TBLPROPERTIES ('kudu.master_addresses'='localhost'){noformat}
> We need to decide on the right behavior for "show create table" and update the tests.
> For Kudu tables, tables with TRANSLATED_TO_EXTERNAL=true and external.table.purge=TRUE should be equivalent to a non-external Kudu table, and we can just detect this case and generate the same SQL as before.
> Other cases may need new logic. I think it makes sense to also address other tests due to MANAGED vs EXTERNAL distinction or extra table properties with this JIRA. Here is a list of tests that seem to have this problem:
> {noformat}
> metadata/test_ddl.py TestDdlStatements.test_create_alter_tbl_properties
> metadata/test_show_create_table.py *
> query_test/test_kudu.py TestShowCreateTable*
> org.apache.impala.catalog.CatalogTest.testCreateTableMetadata
> org.apache.impala.catalog.local.LocalCatalogTest.testKuduTable{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org