You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@kylin.apache.org by "Shaofeng SHI (JIRA)" <ji...@apache.org> on 2015/08/25 04:24:45 UTC

[jira] [Resolved] (KYLIN-651) POC for calculating cube by splits

     [ https://issues.apache.org/jira/browse/KYLIN-651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shaofeng SHI resolved KYLIN-651.
--------------------------------
    Resolution: Fixed

This development of new cubing has been done; Will release in the future release; There is a blog describes this new algorithm:

https://kylin.incubator.apache.org/blog/2015/08/15/fast-cubing/

> POC for calculating cube by splits
> ----------------------------------
>
>                 Key: KYLIN-651
>                 URL: https://issues.apache.org/jira/browse/KYLIN-651
>             Project: Kylin
>          Issue Type: Sub-task
>          Components: General
>            Reporter: Shaofeng SHI
>            Assignee: Shaofeng SHI
>
> This idea is to write a new map-reduce for calculating cube: split the data set to several pieces (feed to mapper); each mapper calculate a cube (with all cuboids) in memory for the given data split; Then send all the cuboids which have the same cuboid id to a reducer; The reducer will merge those cuboids into one big cuboid; Finally write the cuboid into HFile;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)