You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Shaofeng SHI (JIRA)" <ji...@apache.org> on 2018/02/01 10:05:00 UTC
[jira] [Commented] (KYLIN-2932) Simplify the thread model for
in-memory cubing
[ https://issues.apache.org/jira/browse/KYLIN-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348324#comment-16348324 ]
Shaofeng SHI commented on KYLIN-2932:
-------------------------------------
Hi Yanghong,
A couple of questions for this patch:
1. Under what condition this new builder should be used? Can it totally replace the old version?
2. Is there any evidence number to support this improvement? like performance improvement, resource consumption, etc.
3. If this can be covered by integration test, that would be great!
Thanks!
> Simplify the thread model for in-memory cubing
> ----------------------------------------------
>
> Key: KYLIN-2932
> URL: https://issues.apache.org/jira/browse/KYLIN-2932
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
> Reporter: Wang Ken
> Assignee: Wang Ken
> Priority: Major
> Attachments: APACHE-KYLIN-2932.patch
>
>
> The current implementation uses split threads, task threads and main thread to do the cube building, there is complex join and error handling logic.
> The new implement leverages the ForkJoinPool from JDK, the event split logic is handled in
> main thread. Cuboid task and sub-tasks are handled in fork join pool, cube results are collected
> async and can be write to output earlier.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)