You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@kylin.apache.org by "Shuangyin Ge (JIRA)" <ji...@apache.org> on 2018/07/03 03:55:00 UTC

[jira] [Commented] (KYLIN-3435) Only keep base cuboid files on HDFS for future merge

    [ https://issues.apache.org/jira/browse/KYLIN-3435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530787#comment-16530787 ] 

Shuangyin Ge commented on KYLIN-3435:
-------------------------------------

+1 Nice enhancement

> Only keep base cuboid files on HDFS for future merge
> ----------------------------------------------------
>
>                 Key: KYLIN-3435
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3435
>             Project: Kylin
>          Issue Type: Improvement
>          Components: Job Engine
>            Reporter: Shaofeng SHI
>            Priority: Major
>
> Today Kylin keeps all cuboids data in HDFS for future merge. When doing the merge, Kylin need re-encode the dimension values with the new dictionaries, for all cuboids.
>  
> If we only keep the base cuboid, lots of disk space can be saved. On merge, after merge the base cuboid, calculate others from the new base cuboid.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)