You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kudu.apache.org by "Alexey Serbin (Jira)" <ji...@apache.org> on 2021/12/20 21:54:00 UTC
[jira] [Commented] (KUDU-3338) Using the Hive Metastore with Kudu not work
[ https://issues.apache.org/jira/browse/KUDU-3338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17462848#comment-17462848 ]
Alexey Serbin commented on KUDU-3338:
-------------------------------------
Looks like there was some issue at the Hive/HMS side, nothing seems to point at something at the Kudu side. I guess next logical step would be checking Hive/HMS logs to figure out what went wrong there.
There were reports with similar error messages with HMS {{kudu}} CLI tool when working with Hive/HMS that included the fix for HIVE-24920. Reverting [the corresponding Hive commit|https://github.com/apache/hive/commit/33dfe71ab91e646e4624043d2302701ac29c36db] helped to resolve the problem.
Not sure that's exactly the culprit in your case, but you can check if you see similar problem if using the {{kudu}} CLI tool to create a table once HMS integration is enabled:
{noformat}
sudo -u kudu kudu perf loadgen -keep_auto_table -table_num_hash_partitions=8 -table_num_replicas=3 <kudu_master_RPC_address>
Command exited with code: 1
Command output: Remote error: Error creating table default.loadgen_auto_83fcd3766e1b4feb9c64572ecc5b6bbb on the master: failed to create HMS catalog entry for table [id=ec0295b35aae45ef9c12c93719f448fc]: failed to create Hive MetaStore table: TException - service has thrown: MetaException(message=java.lang.IllegalArgumentException: Can not create a Path from an empty string)
{noformat}
If running the above (substitute the address of Kudu master as needed) produces similar error in your case, I bet rolling back [33dfe71ab91e646e4624043d2302701ac29c36db|https://github.com/apache/hive/commit/33dfe71ab91e646e4624043d2302701ac29c36db] will help.
> Using the Hive Metastore with Kudu not work
> -------------------------------------------
>
> Key: KUDU-3338
> URL: https://issues.apache.org/jira/browse/KUDU-3338
> Project: Kudu
> Issue Type: Bug
> Components: client
> Affects Versions: 1.15.0
> Environment: kudu：kudu 1.15.0 (rev non-git-build)
> hive： Apache Hive (version 3.1.0.3.1.0.0-78)
> hdp： 3.1.1.3.1
> trino：364
> Reporter: liu
> Priority: Critical
> Fix For: NA
>
>
> I follow this page config . It looks as if the configuration was successful.
> [https://kudu.apache.org/docs/hive_metastore.html#enabling-the-hive-metastore-integration]
> kudu master start log
> {code:java}
> I1115 18:51:37.391942 1832 catalog_manager.cc:1253] Loading table and tablet metadata into memory...
> I1115 18:51:37.392135 1832 catalog_manager.cc:495] Loaded metadata for table $schemas [id=9c31d249228f42b38468835a7ae2c6e6]
> I1115 18:51:37.392266 1832 catalog_manager.cc:549] Loaded metadata for tablet 1526622b192145b8973fc852c2cfbd8f (table $schemas [id=9c31d249228f42b38468835a7ae2c6e6])
> I1115 18:51:37.392287 1832 catalog_manager.cc:549] Loaded metadata for tablet 2842be87bec74f0592a01ca0535bd9aa (table $schemas [id=9c31d249228f42b38468835a7ae2c6e6])
> I1115 18:51:37.392294 1832 catalog_manager.cc:1262] Initializing Kudu cluster ID...
> I1115 18:51:37.392381 1832 catalog_manager.cc:1098] Loaded cluster ID: 70b19944b04543759922355e6ce259ac
> I1115 18:51:37.392387 1832 catalog_manager.cc:1273] Initializing Kudu internal certificate authority...
> I1115 18:51:37.392593 1832 catalog_manager.cc:1282] Loading token signing keys...
> I1115 18:51:37.392693 1832 catalog_manager.cc:5093] T 00000000000000000000000000000000 P 2bc3b2318ca640a78a99fcbe4d058a9f: Loaded TSK: 0
> I1115 18:51:37.392736 1832 catalog_manager.cc:1292] Initializing in-progress tserver states...
> I1115 18:51:37.392812 1832 catalog_manager.cc:1305] Loading latest processed Hive Metastore notification log event ID...{code}
> Now I use trino to connect to kudu and execute the following script。
> {code:java}
> trino:default> create schema cdr;
> CREATE SCHEMA
> trino:default> use cdr;
> USE
> trino:cdr> show schemas;
> Schema
> --------------------
> cdr
> default
> information_schema
> (3 rows)Query 20211118_033415_00020_4gwuw, FINISHED, 3 nodes
> Splits: 36 total, 36 done (100.00%)
> 0.22 [3 rows, 43B] [13 rows/s, 195B/s]
> trino:cdr> CREATE TABLE kudu.cdr.users (
> -> user_id int WITH (primary_key = true),
> -> first_name varchar,
> -> last_name varchar
> -> ) WITH (
> -> partition_by_hash_columns = ARRAY['user_id'],
> -> partition_by_hash_buckets = 2
> -> );
> ->
> W1118 13:56:00.671370 31226 catalog_manager.cc:1959] Remote error: failed to create HMS catalog entry for table [id=3490249b929842509d3364a18f07a4e5]: failed to create Hive MetaStore table: TException - service has thrown: MetaException(message=NoSuchObjectException(message:cdr)) {code}
> master log：
> {code:java}
> W1118 13:56:00.671370 31226 catalog_manager.cc:1959] Remote error: failed to create HMS catalog entry for table [id=3490249b929842509d3364a18f07a4e5]: failed to create Hive MetaStore table: TException - service has thrown: MetaException(message=NoSuchObjectException(message:cdr)) {code}
> Schemas Failed to synchronize to hive metadata，If I create this database in hive,the error log：
> {code:java}
> W1118 13:30:00.148990 31226 catalog_manager.cc:1959] Remote error: failed to create HMS catalog entry for table [id=4a40e0c12d9a4d26a11fcce0cf259d35]: failed to create Hive MetaStore table: TException - service has thrown: MetaException(message=java.lang.IllegalArgumentException: Can not create a Path from an empty string) {code}
> Now I don't know how to solve this problem
--
This message was sent by Atlassian Jira
(v8.20.1#820001)