You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by Joel Victor <jo...@gmail.com> on 2016/10/27 18:09:38 UTC
Build times gone up!
We are seeing a problem in our cube builds that our build times have gone
up. In between two steps we see that it is spending 30 seconds. Our cube
build is a in memory cube build and while going through step #6 to step #17
it takes ~ 5.5 minutes. I would like to know what happens in between these
steps? From the logs I can see that it is updating metadata in HBase. Is
there anything else other than HBase update that is being done in between
steps? I have attached the log statements for a few steps.
Re: Build times gone up!
Posted by ShaoFeng Shi <sh...@apache.org>.
As far as I know, there is no other operations, just updating each step's
output to meta store (hbase); usually it moves fast if layered cubing steps
are skipped;
2016-10-28 10:16 GMT+08:00 Joel Victor <jo...@gmail.com>:
> Hello Billy,
>
> I understand that. My question is why does it take 30 seconds between
> skipping steps ? And what happens when skipping steps ?
>
> On Fri, Oct 28, 2016 at 6:14 AM, Billy(Yiming) Liu <
> liuyiming.vip@gmail.com> wrote:
>
>> Hi Joel,
>>
>> If memory build algorithm is used, the steps for layered build would be
>> skipped, such like "Build N-Dimension". All build actually happens at "Build
>> Cube".
>>
>> 2016-10-28 2:09 GMT+08:00 Joel Victor <jo...@gmail.com>:
>>
>>> We are seeing a problem in our cube builds that our build times have
>>> gone up. In between two steps we see that it is spending 30 seconds. Our
>>> cube build is a in memory cube build and while going through step #6 to
>>> step #17 it takes ~ 5.5 minutes. I would like to know what happens in
>>> between these steps? From the logs I can see that it is updating metadata
>>> in HBase. Is there anything else other than HBase update that is being done
>>> in between steps? I have attached the log statements for a few steps.
>>>
>>
>>
>>
>> --
>> With Warm regards
>>
>> Yiming Liu (刘一鸣)
>>
>
>
--
Best regards,
Shaofeng Shi 史少锋
Re: Build times gone up!
Posted by Joel Victor <jo...@gmail.com>.
Hello Billy,
I understand that. My question is why does it take 30 seconds between
skipping steps ? And what happens when skipping steps ?
On Fri, Oct 28, 2016 at 6:14 AM, Billy(Yiming) Liu <li...@gmail.com>
wrote:
> Hi Joel,
>
> If memory build algorithm is used, the steps for layered build would be
> skipped, such like "Build N-Dimension". All build actually happens at "Build
> Cube".
>
> 2016-10-28 2:09 GMT+08:00 Joel Victor <jo...@gmail.com>:
>
>> We are seeing a problem in our cube builds that our build times have gone
>> up. In between two steps we see that it is spending 30 seconds. Our cube
>> build is a in memory cube build and while going through step #6 to step #17
>> it takes ~ 5.5 minutes. I would like to know what happens in between these
>> steps? From the logs I can see that it is updating metadata in HBase. Is
>> there anything else other than HBase update that is being done in between
>> steps? I have attached the log statements for a few steps.
>>
>
>
>
> --
> With Warm regards
>
> Yiming Liu (刘一鸣)
>
Re: Build times gone up!
Posted by "Billy(Yiming) Liu" <li...@gmail.com>.
Hi Joel,
If memory build algorithm is used, the steps for layered build would be
skipped, such like "Build N-Dimension". All build actually happens at "Build
Cube".
2016-10-28 2:09 GMT+08:00 Joel Victor <jo...@gmail.com>:
> We are seeing a problem in our cube builds that our build times have gone
> up. In between two steps we see that it is spending 30 seconds. Our cube
> build is a in memory cube build and while going through step #6 to step #17
> it takes ~ 5.5 minutes. I would like to know what happens in between these
> steps? From the logs I can see that it is updating metadata in HBase. Is
> there anything else other than HBase update that is being done in between
> steps? I have attached the log statements for a few steps.
>
--
With Warm regards
Yiming Liu (刘一鸣)