Kylin ingnores startTime in cube build process

[Marek Wiewiorka <ma...@gmail.com>]

Hi All,
I came across a weird situation with Kylin cube incremental refresh.
I will try to describe it:
I've a cube with data up to 2015.12.03.
When tried to to build a cube segment for 2015.12.06 by specify a json like
this:
{
 "startTime": "1449360000000",
 "endTime": "1449446400000",
 "buildType": "BUILD"
}
Kylin ignored startTime and started building a segment 2015-12-03 --
2015-12-07
instead 2015-12-06 -- 2015-12-07 - so starting to build a segment starting
from the most
recent date in segment already available not the one I specified in a json
file.

So is it a bug or a feature that it's not possible to build a one-day
segment in such a scenario - it seems like Kylin does not allow gaps
between segments?

Thanks in advance!

Marek

Re: Kylin ingnores startTime in cube build process

[Li Yang <li...@apache.org>]

What  Marek  described is as designed (an early design though, and we can
discuss if the design makes sense).

Out of simplicity, Kylin 1.x requires all segment be continuous and there's
no gap in between. So a new incremental segment always starts from where
the last segment ends. I agree the Rest API should not modify the start
time without warn the user. Pls open a JIRA for it.

>From Kylin 2.x, there can be holes between segments. However I need to
double check if the Rest API was updated to respect the input start time.

On Tue, Dec 8, 2015 at 12:16 PM, 周千昊 <z....@gmail.com> wrote:

> Hi Marek
>      Is this your first time to build the cube?
>      Kylin will use the end time of last segment's as the start time. And
> if there are no segments have been built, the partition start date will be
> used as the first segment's start time.
>
> Marek Wiewiorka <ma...@gmail.com>于2015年12月7日周一 下午8:14写道：
>
> > Hi All,
> > I came across a weird situation with Kylin cube incremental refresh.
> > I will try to describe it:
> > I've a cube with data up to 2015.12.03.
> > When tried to to build a cube segment for 2015.12.06 by specify a json
> like
> > this:
> > {
> >  "startTime": "1449360000000",
> >  "endTime": "1449446400000",
> >  "buildType": "BUILD"
> > }
> > Kylin ignored startTime and started building a segment 2015-12-03 --
> > 2015-12-07
> > instead 2015-12-06 -- 2015-12-07 - so starting to build a segment
> starting
> > from the most
> > recent date in segment already available not the one I specified in a
> json
> > file.
> >
> > So is it a bug or a feature that it's not possible to build a one-day
> > segment in such a scenario - it seems like Kylin does not allow gaps
> > between segments?
> >
> > Thanks in advance!
> >
> > Marek
> >
> --
> Best Regard
> ZhouQianhao
>

Re: Kylin ingnores startTime in cube build process

[周千昊 <z....@gmail.com>]

Hi Marek
     Is this your first time to build the cube?
     Kylin will use the end time of last segment's as the start time. And
if there are no segments have been built, the partition start date will be
used as the first segment's start time.

Marek Wiewiorka <ma...@gmail.com>于2015年12月7日周一 下午8:14写道：

> Hi All,
> I came across a weird situation with Kylin cube incremental refresh.
> I will try to describe it:
> I've a cube with data up to 2015.12.03.
> When tried to to build a cube segment for 2015.12.06 by specify a json like
> this:
> {
>  "startTime": "1449360000000",
>  "endTime": "1449446400000",
>  "buildType": "BUILD"
> }
> Kylin ignored startTime and started building a segment 2015-12-03 --
> 2015-12-07
> instead 2015-12-06 -- 2015-12-07 - so starting to build a segment starting
> from the most
> recent date in segment already available not the one I specified in a json
> file.
>
> So is it a bug or a feature that it's not possible to build a one-day
> segment in such a scenario - it seems like Kylin does not allow gaps
> between segments?
>
> Thanks in advance!
>
> Marek
>
-- 
Best Regard
ZhouQianhao