You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by "wangdan@snqu.com" <wa...@snqu.com> on 2016/11/30 08:02:58 UTC
回复: merge
hi,
the merge information is :
PARTITION DATE COLUMN DEFAULT.DMP_USER.TIMESTAMP
MERGE START SEGMENT 20161110000000-20161120000000
MERGE END SEGMENT 20161120000000-20161130000000
START SEGMENT DETAIL Start Date(Include) 2016-11-10 00:00:00
End Date(Exclude) 2016-11-20 00:00:00
Last build Time 2016-11-30 14:18:49 GMT+8
Last build ID b89bfb86-0395-4d55-8f8f-d25df3f59fdc
END SEGMENT DETAIL Start Date(Include) 2016-11-20 00:00:00
End Date(Exclude) 2016-11-30 00:00:00
Last build Time 2016-11-30 14:24:32 GMT+8
Last build ID 26a55d70-9872-42de-8d93-96949a342729
wangdan@snqu.com
发件人： wangdan@snqu.com
发送时间： 2016-11-30 15:28
收件人： dev
主题： merge
Hi,
I have a problem about merge
when I do merge, the information is following:
the two segments are build from the same cube and one same table, after click "submit' I got
“empty cube segment found:[20161120000000_20161130000000, do you want to merge segments forcely?”
wangdan@snqu.com
Re: Re: merge
Posted by "wangdan@snqu.com" <wa...@snqu.com>.
thanks!
I've found the reason. The MERGE END SEGMENT 20161120000000-20161130000000 if empty.
when build MERGE END SEGMENT, there are some rows of date 20161130 in table, but the MERGE END SEGMENT build from rows between [20161120, 20161130),
so the MERGE END SEGMENT is empty.
wangdan@snqu.com
From: ShaoFeng Shi
Date: 2016-12-01 10:19
To: dev
Subject: Re: merge
This warning is to remind user that there is one segment which has 0
records, we also call it as "empty" segment;
Empty segment is allowed in Kylin by default; because it is possible that
for a given time period there is no record in source table (especially in
stream + tiny time window case);
But empty segment might not be expected; it may indicate something went
wrong in upstream work flow. User need to investigate and then refresh the
segment. If you want Kylin fail the build job when there is 0 records, set "
kylin.job.allow.empty.segment=false" in conf/kylin.properties
After merge the empty segment with other segment, you will not be able to
independently refresh that small time period; you have to refresh with the
merged time period, which will cost more resource, that's why Kylin remind
user here; If you think the empty segment is okay, you can forcely merge
them.
2016-11-30 16:02 GMT+08:00 wangdan@snqu.com <wa...@snqu.com>:
> hi,
> the merge information is :
>
> PARTITION DATE COLUMN DEFAULT.DMP_USER.TIMESTAMP
> MERGE START SEGMENT 20161110000000-20161120000000
> MERGE END SEGMENT 20161120000000-20161130000000
> START SEGMENT DETAIL Start Date(Include) 2016-11-10 00:00:00
> End Date(Exclude) 2016-11-20 00:00:00
> Last build Time 2016-11-30 14:18:49 GMT+8
> Last build ID b89bfb86-0395-4d55-8f8f-d25df3f59fdc
> END SEGMENT DETAIL Start Date(Include) 2016-11-20 00:00:00
> End Date(Exclude) 2016-11-30 00:00:00
> Last build Time 2016-11-30 14:24:32 GMT+8
> Last build ID 26a55d70-9872-42de-8d93-96949a342729
>
>
>
>
>
> wangdan@snqu.com
>
> 发件人： wangdan@snqu.com
> 发送时间： 2016-11-30 15:28
> 收件人： dev
> 主题： merge
> Hi,
> I have a problem about merge
> when I do merge, the information is following:
> the two segments are build from the same cube and one same table, after
> click "submit' I got
> “empty cube segment found:[20161120000000_20161130000000, do you want to
> merge segments forcely?”
>
>
>
> wangdan@snqu.com
>
--
Best regards,
Shaofeng Shi 史少锋
Re: merge
Posted by ShaoFeng Shi <sh...@apache.org>.
This warning is to remind user that there is one segment which has 0
records, we also call it as "empty" segment;
Empty segment is allowed in Kylin by default; because it is possible that
for a given time period there is no record in source table (especially in
stream + tiny time window case);
But empty segment might not be expected; it may indicate something went
wrong in upstream work flow. User need to investigate and then refresh the
segment. If you want Kylin fail the build job when there is 0 records, set "
kylin.job.allow.empty.segment=false" in conf/kylin.properties
After merge the empty segment with other segment, you will not be able to
independently refresh that small time period; you have to refresh with the
merged time period, which will cost more resource, that's why Kylin remind
user here; If you think the empty segment is okay, you can forcely merge
them.
2016-11-30 16:02 GMT+08:00 wangdan@snqu.com <wa...@snqu.com>:
> hi,
> the merge information is :
>
> PARTITION DATE COLUMN DEFAULT.DMP_USER.TIMESTAMP
> MERGE START SEGMENT 20161110000000-20161120000000
> MERGE END SEGMENT 20161120000000-20161130000000
> START SEGMENT DETAIL Start Date(Include) 2016-11-10 00:00:00
> End Date(Exclude) 2016-11-20 00:00:00
> Last build Time 2016-11-30 14:18:49 GMT+8
> Last build ID b89bfb86-0395-4d55-8f8f-d25df3f59fdc
> END SEGMENT DETAIL Start Date(Include) 2016-11-20 00:00:00
> End Date(Exclude) 2016-11-30 00:00:00
> Last build Time 2016-11-30 14:24:32 GMT+8
> Last build ID 26a55d70-9872-42de-8d93-96949a342729
>
>
>
>
>
> wangdan@snqu.com
>
> 发件人： wangdan@snqu.com
> 发送时间： 2016-11-30 15:28
> 收件人： dev
> 主题： merge
> Hi,
> I have a problem about merge
> when I do merge, the information is following:
> the two segments are build from the same cube and one same table, after
> click "submit' I got
> “empty cube segment found:[20161120000000_20161130000000, do you want to
> merge segments forcely?”
>
>
>
> wangdan@snqu.com
>
--
Best regards,
Shaofeng Shi 史少锋