[jira] [Created] (KYLIN-4002) Cleanup job does not delete Hive intermediate VIEW tables

["Iñigo Martinez (JIRA)" <ji...@apache.org>]

Iñigo Martinez created KYLIN-4002:
-------------------------------------

             Summary: Cleanup job does not delete Hive intermediate VIEW tables 
                 Key: KYLIN-4002
                 URL: https://issues.apache.org/jira/browse/KYLIN-4002
             Project: Kylin
          Issue Type: Bug
          Components: Job Engine
    Affects Versions: v2.6.1
            Reporter: Iñigo Martinez


Kylin org.apache.kylin.tool.StorageCleanupJob does not delete hive intermediate tables if they are view based.

After several weeks running Kylin 2.6.1 after a successful upgrade from 2.4.1, we have a lot of temporary tables that have not been deleted. All of them come from a VIEW.

However, those tables built directly from a table and not from a view, are properly deleted.

Tables from view not deleted: 263079

Tables not form view not deleted: 0

 

At log level, we can see:
{code:java}
2019-05-14 08:10:00,293 DEBUG [main] job.StorageCleanupJob:326 : Checking if table is garbage -- kylin_intermediate_dw_di_paises_view_238e3004_13e8_2be6_5f0e_df545b481986
2019-05-14 10:17:53,235 INFO [main] job.StorageCleanupJob:407 : Deleting Hive table kylin_intermediate_dw_di_paises_view_238e3004_13e8_2be6_5f0e_df545b481986
drop table if exists kylin_intermediate_dw_di_paises_view_238e3004_13e8_2be6_5f0e_df545b481986;{code}
So, it's properly detected and drop sentence properly created. But it's not executed and table is still there.

We have launched the 263079 drop sentences via script and all are being removed.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)