[jira] [Commented] (KYLIN-4679) StorageCleanupJob clean hive table support hive table prefix

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KYLIN-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212412#comment-17212412 ] 

ASF subversion and git services commented on KYLIN-4679:
--------------------------------------------------------

Commit 39755345f6af35747c505a8e7e49282aaaaea1b9 in kylin's branch refs/heads/master-hadoop3 from chuxiao
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=3975534 ]

KYLIN-4679 clean hive table support hive table prefix


> StorageCleanupJob clean hive table support hive table prefix
> ------------------------------------------------------------
>
>                 Key: KYLIN-4679
>                 URL: https://issues.apache.org/jira/browse/KYLIN-4679
>             Project: Kylin
>          Issue Type: Improvement
>            Reporter: chuxiao
>            Assignee: chuxiao
>            Priority: Major
>             Fix For: v3.1.1
>
>
> 我的库有几千张表。StorageCleanupJob的行为是获取库里所有表的元数据再过滤表名。首先，取所有表元数据的行为会给metastore较大压力。其次，我们的metastore当一次取超过1千张表的元数据，会直接报错。所以需要增加提前用表名前缀过滤数据的功能。
> My database has thousands of tables. StorageCleanupJob's behavior is to get the metadata of all tables in the database and then filter table name. 
> First, the behavior of getting all tables metadata will give metastore great pressure. Secondly,  when getting metadata of more than 1,000 tables at one time, our metastore would directly return errors. So it needs to add the function of filtering data with table name prefix in advance



--
This message was sent by Atlassian Jira
(v8.3.4#803005)