You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@kylin.apache.org by "liyang (JIRA)" <ji...@apache.org> on 2015/08/25 04:30:45 UTC

[jira] [Resolved] (KYLIN-781) Star Cubing alike partial cuboid flow down

     [ https://issues.apache.org/jira/browse/KYLIN-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

liyang resolved KYLIN-781.
--------------------------
       Resolution: Later
    Fix Version/s:     (was: v0.8.1)

Now dogged in-mem cubing is flexible to mem footprint. this can be postponed to later.

> Star Cubing alike partial cuboid flow down
> ------------------------------------------
>
>                 Key: KYLIN-781
>                 URL: https://issues.apache.org/jira/browse/KYLIN-781
>             Project: Kylin
>          Issue Type: Sub-task
>          Components: Job Engine
>            Reporter: liyang
>            Assignee: liyang
>
> Key idea
> - Assume base cuboid fit in mem (for a mapper split of course)
> - For child cuboid having common dim prefix, flow down records to child as soon as completed records can be fulfilled in child
> - Keep completed records in mem as long as possible, but also save them to disk in the background
> - When mem run short, pause flow down, remove completed records from mem, and resume flow from disk
> - Multiple flow downs at the same time, configurable thread number



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)