You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kylin.apache.org by "Shaofeng SHI (JIRA)" <ji...@apache.org> on 2018/07/01 12:23:00 UTC

[jira] [Created] (KYLIN-3435) Only keep base cuboid files on HDFS for future merge

Shaofeng SHI created KYLIN-3435:
-----------------------------------

             Summary: Only keep base cuboid files on HDFS for future merge
                 Key: KYLIN-3435
                 URL: https://issues.apache.org/jira/browse/KYLIN-3435
             Project: Kylin
          Issue Type: Improvement
          Components: Job Engine
            Reporter: Shaofeng SHI


Today Kylin keeps all cuboids data in HDFS for future merge. When doing the merge, Kylin need re-encode the dimension values with the new dictionaries, for all cuboids.

 

If we only keep the base cuboid, lots of disk space can be saved. On merge, after merge the base cuboid, calculate others from the new base cuboid.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)