You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "WangSheng (Jira)" <ji...@apache.org> on 2020/10/30 09:09:00 UTC
[jira] [Updated] (KYLIN-4802) “Build N-Dimension Cuboid” execute twice when using DistributedScheduler
[ https://issues.apache.org/jira/browse/KYLIN-4802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
WangSheng updated KYLIN-4802:
-----------------------------
Description:
I met a problem when using DistributedScheduler in two nodes, my current cluster version is 2.6.6. When executing "Build N-Dimension Cuboid : level 4" step, I found this step submitted MR job in both two nodes. One node submitted a MR first, and then executed following steps, when executing "Convert Cuboid Data to HFile" step, another node submitted a MR for "Build N-Dimension Cuboid : level 4" step again. And this caused data missing when generated File. And after this job completed, query on level 4 returns empty.
In the picture, we can found that "Build N-Dimension Cuboid : level 4" step finished time is later than "Convert Cuboid Data to HFile" step start time:
!kylin01.png|width=264,height=280!
And these two pictures are from two nodes' Kylin.log, we can found that two MR jobs are submitted for step 10, which is "Build N-Dimension Cuboid : level 4" step:
!kylin02.png|width=324,height=78!
!kylin03.png|width=375,height=164!
was:I met a problem when using DistributedScheduler in two node, my current cluster version is 2.6.6. When executing "Build N-Dimension Cuboid : level 4" step, I found this step submitted MR job in both two nodes. One node submitted a MR first, and then executed following steps, when executing "Convert Cuboid Data to HFile" step, another node submitted a MR for "Build N-Dimension Cuboid : level 4" step again. And this caused data missing when generated File. And after this job completed, query on level 4 returns empty.
> “Build N-Dimension Cuboid” execute twice when using DistributedScheduler
> ------------------------------------------------------------------------
>
> Key: KYLIN-4802
> URL: https://issues.apache.org/jira/browse/KYLIN-4802
> Project: Kylin
> Issue Type: Bug
> Affects Versions: v2.6.6
> Reporter: WangSheng
> Priority: Major
> Attachments: kylin01.png, kylin02.png, kylin03.png
>
>
> I met a problem when using DistributedScheduler in two nodes, my current cluster version is 2.6.6. When executing "Build N-Dimension Cuboid : level 4" step, I found this step submitted MR job in both two nodes. One node submitted a MR first, and then executed following steps, when executing "Convert Cuboid Data to HFile" step, another node submitted a MR for "Build N-Dimension Cuboid : level 4" step again. And this caused data missing when generated File. And after this job completed, query on level 4 returns empty.
> In the picture, we can found that "Build N-Dimension Cuboid : level 4" step finished time is later than "Convert Cuboid Data to HFile" step start time:
> !kylin01.png|width=264,height=280!
> And these two pictures are from two nodes' Kylin.log, we can found that two MR jobs are submitted for step 10, which is "Build N-Dimension Cuboid : level 4" step:
> !kylin02.png|width=324,height=78!
> !kylin03.png|width=375,height=164!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)