You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Xiaoxiang Yu (Jira)" <ji...@apache.org> on 2020/07/10 11:40:00 UTC
[jira] [Closed] (KYLIN-4520) Improve StorageCleanJobHbaseUtil to
cleanup HBase tables in parallel
[ https://issues.apache.org/jira/browse/KYLIN-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xiaoxiang Yu closed KYLIN-4520.
-------------------------------
Resolved in release 3.1.0 (2020-07-03)
> Improve StorageCleanJobHbaseUtil to cleanup HBase tables in parallel
> --------------------------------------------------------------------
>
> Key: KYLIN-4520
> URL: https://issues.apache.org/jira/browse/KYLIN-4520
> Project: Kylin
> Issue Type: Improvement
> Components: Tools, Build and Test
> Reporter: Shao Feng Shi
> Assignee: Shao Feng Shi
> Priority: Major
> Fix For: v3.1.0
>
>
> The current StorageCleanJobHbaseUtil supports multi-threads, while from the execution logs, it still check one by one, so not really multi-threading.
>
> Below is the logs that use 5 threads " -- thread 5":
>
> {code:java}
> // code placeholder
> 2020-05-23 02:02:26,538 INFO [main] job.StorageCleanJobHbaseUtil:114 : Use 5 threads to drop unused hbase tables
> 2020-05-23 02:02:26,539 INFO [pool-5-thread-1] job.StorageCleanJobHbaseUtil:171 : Deleting HBase table KYLIN_56XU9WDLOZ
> 2020-05-23 02:02:26,562 INFO [pool-5-thread-1] job.StorageCleanJobHbaseUtil:180 : HBase tableKYLIN_56XU9WDLOZ does not exist
> 2020-05-23 02:02:26,564 INFO [pool-5-thread-2] job.StorageCleanJobHbaseUtil:171 : Deleting HBase table KYLIN_590IHJ5QTM
> 2020-05-23 02:02:26,576 INFO [pool-5-thread-2] job.StorageCleanJobHbaseUtil:180 : HBase tableKYLIN_590IHJ5QTM does not exist
> 2020-05-23 02:02:26,577 INFO [pool-5-thread-3] job.StorageCleanJobHbaseUtil:171 : Deleting HBase table KYLIN_7U8J7ZFBYN
> 2020-05-23 02:02:26,586 INFO [pool-5-thread-3] job.StorageCleanJobHbaseUtil:180 : HBase tableKYLIN_7U8J7ZFBYN does not exist
> 2020-05-23 02:02:26,587 INFO [pool-5-thread-4] job.StorageCleanJobHbaseUtil:171 : Deleting HBase table KYLIN_BOR03PP4PV
> 2020-05-23 02:02:26,595 INFO [pool-5-thread-4] job.StorageCleanJobHbaseUtil:180 : HBase tableKYLIN_BOR03PP4PV does not exist
> 2020-05-23 02:02:26,596 INFO [pool-5-thread-5] job.StorageCleanJobHbaseUtil:171 : Deleting HBase table KYLIN_ESGGE02OQI
> 2020-05-23 02:02:26,606 INFO [pool-5-thread-5] job.StorageCleanJobHbaseUtil:180 : HBase tableKYLIN_ESGGE02OQI does not exist
> 2020-05-23 02:02:26,606 INFO [pool-5-thread-1] job.StorageCleanJobHbaseUtil:171 : Deleting HBase table KYLIN_FHWPLEOSO6
> 2020-05-23 02:02:26,617 INFO [pool-5-thread-1] job.StorageCleanJobHbaseUtil:180 : HBase tableKYLIN_FHWPLEOSO6 does not exist
> 2020-05-23 02:02:26,618 INFO [pool-5-thread-2] job.StorageCleanJobHbaseUtil:171 : Deleting HBase table KYLIN_GWYKX70W9A
> 2020-05-23 02:02:26,642 INFO [pool-5-thread-2] client.HBaseAdmin:1330 : Started disable of KYLIN_GWYKX70W9A
> 2020-05-23 02:02:30,802 INFO [pool-5-thread-2] client.HBaseAdmin:1357 : Disabled KYLIN_GWYKX70W9A
> 2020-05-23 02:02:32,094 INFO [pool-5-thread-2] client.HBaseAdmin:924 : Deleted KYLIN_GWYKX70W9A
> 2020-05-23 02:02:32,094 INFO [pool-5-thread-2] job.StorageCleanJobHbaseUtil:178 : Deleted HBase table KYLIN_GWYKX70W9A
> 2020-05-23 02:02:32,095 INFO [pool-5-thread-3] job.StorageCleanJobHbaseUtil:171 : Deleting HBase table KYLIN_HLB7508SUV
> 2020-05-23 02:02:32,113 INFO [pool-5-thread-3] client.HBaseAdmin:1330 : Started disable of KYLIN_HLB7508SUV
> 2020-05-23 02:02:34,348 INFO [pool-5-thread-3] client.HBaseAdmin:1357 : Disabled KYLIN_HLB7508SUV
> 2020-05-23 02:02:36,582 INFO [pool-5-thread-3] client.HBaseAdmin:924 : Deleted KYLIN_HLB7508SUV
> 2020-05-23 02:02:36,582 INFO [pool-5-thread-3] job.StorageCleanJobHbaseUtil:178 : Deleted HBase table KYLIN_HLB7508SUV
> 2020-05-23 02:02:36,582 INFO [pool-5-thread-4] job.StorageCleanJobHbaseUtil:171 : Deleting HBase table KYLIN_JPIJ7GT1F5
> 2020-05-23 02:02:36,599 INFO [pool-5-thread-4] client.HBaseAdmin:1330 : Started disable of KYLIN_JPIJ7GT1F5
> 2020-05-23 02:02:40,959 INFO [pool-5-thread-4] client.HBaseAdmin:1357 : Disabled KYLIN_JPIJ7GT1F5
> 2020-05-23 02:02:42,119 INFO [pool-5-thread-4] client.HBaseAdmin:924 : Deleted KYLIN_JPIJ7GT1F5
> 2020-05-23 02:02:42,119 INFO [pool-5-thread-4] job.StorageCleanJobHbaseUtil:178 : Deleted HBase table KYLIN_JPIJ7GT1F5
> 2020-05-23 02:02:42,119 INFO [pool-5-thread-5] job.StorageCleanJobHbaseUtil:171 : Deleting HBase table KYLIN_KV4U23OBEP
> 2020-05-23 02:02:42,141 INFO [pool-5-thread-5] client.HBaseAdmin:1330 : Started disable of KYLIN_KV4U23OBEP
> 2020-05-23 02:02:46,322 INFO [pool-5-thread-5] client.HBaseAdmin:1357 : Disabled KYLIN_KV4U23OBEP
> 2020-05-23 02:02:47,958 INFO [pool-5-thread-5] client.HBaseAdmin:924 : Deleted KYLIN_KV4U23OBEP
> 2020-05-23 02:02:47,958 INFO [pool-5-thread-5] job.StorageCleanJobHbaseUtil:178 : Deleted HBase table KYLIN_KV4U23OBEP
> 2020-05-23 02:02:47,958 INFO [pool-5-thread-1] job.StorageCleanJobHbaseUtil:171 : Deleting HBase table KYLIN_L802CS1ZZU
> 2020-05-23 02:02:47,975 INFO [pool-5-thread-1] client.HBaseAdmin:1330 : Started disable of KYLIN_L802CS1ZZU
> 2020-05-23 02:02:50,194 INFO [pool-5-thread-1] client.HBaseAdmin:1357 : Disabled KYLIN_L802CS1ZZU
> 2020-05-23 02:02:51,554 INFO [pool-5-thread-1] client.HBaseAdmin:924 : Deleted KYLIN_L802CS1ZZU
> 2020-05-23 02:02:51,555 INFO [pool-5-thread-1] job.StorageCleanJobHbaseUtil:178 : Deleted HBase table KYLIN_L802CS1ZZU
> 2020-05-23 02:02:51,555 INFO [pool-5-thread-2] job.StorageCleanJobHbaseUtil:171 : Deleting HBase table KYLIN_L9D9CZXZIG
> 2020-05-23 02:02:51,570 INFO [pool-5-thread-2] client.HBaseAdmin:1330 : Started disable of KYLIN_L9D9CZXZIG
> 2020-05-23 02:02:53,773 INFO [pool-5-thread-2] client.HBaseAdmin:1357 : Disabled KYLIN_L9D9CZXZIG
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)