Hierarchy Dimension

[Patrick McAnneny <pa...@leadkarma.com>]

The way I understand the hierarchy feature (lets say with three columns
specified) is the following: "create a roll-up aggregate grouping by this
column, then that column, then that column" Is this correct understanding?

 If I would want to query potentially based on any combination of those
three roll-ups, would it be best to create three separate hierarchy
dimensions or the combination of each? Would order even matter here? Order
does not usually matter in 'group by' statements

Re: Hierarchy Dimension

[Luke Han <lu...@gmail.com>]

I see, that make sense, we are trying to add more context and tutorial
samples there:)


Best Regards!
---------------------

Luke Han

On Wed, Sep 23, 2015 at 3:33 AM, Patrick McAnneny <
patrick.mcanneny@leadkarma.com> wrote:

> Agreed, but I'm saying more "what kylin does" rather than "what does this
> term mean". For example, some trail and error is necessary for figuring out
> various UI elements. But I'm sure all of this will improve over time and
> the majority of the UI is great. Thanks!
>
> On Mon, Sep 21, 2015 at 9:17 PM, Luke Han <lu...@gmail.com> wrote:
>
> > As today's DW/BI world, Hierarchy Dimension is still very popular and
> > common for BI experts and users, like most of traditional and modern
> > products doing.
> > I don't think it will confuse users especially when they design cubes.
> >
> > So keep the same concept is very important for user to adopt new stuff,
> > here is Kylin, on new tech stack.
> >
> > Whatever backend setting will be, the user facing term, concept should
> > always keep same with existing one as much as possible:)
> >
> > For documentation, there's terminology page would like to recommend
> > newbie to go through first:
> > http://kylin.incubator.apache.org/docs/gettingstarted/terminology.html
> >
> > There's some enhancement of current UI and tutorial documentation
> > will come, but I don't think it make sense for a project to draft all
> basic
> > and common concept and terms everywhere:-)
> >
> > Thanks.
> >
> >
> > Best Regards!
> > ---------------------
> >
> > Luke Han
> >
> > On Tue, Sep 22, 2015 at 4:15 AM, Patrick McAnneny <
> > patrick.mcanneny@leadkarma.com> wrote:
> >
> > > I found it confusing initially until I read about how it is mostly a
> > method
> > > of optimising the cube (rather than a separate type of dimension that
> > would
> > > be queried differently). It makes a lot of sense now that I understand,
> > but
> > > perhaps a little bit more documentation would help others starting out.
> > (it
> > > is not covered in the cube construction slides)
> > >
> > > Not a huge deal though
> > >
> > > On Mon, Sep 21, 2015 at 3:21 PM, Julian Hyde <jh...@apache.org> wrote:
> > >
> > > > Yes, hierarchies may be a useful organizing concept. I claim that
> > > > hierarchies are separate things: (1) navigation paths for a UI, (2)
> > > > statistical patterns in data access. Conversations like this one are
> > > > evidence that — at least for some people — hierarchies are confusing.
> > So,
> > > > keep them as a user concept only if they make life simpler for the
> > > majority
> > > > of users.
> > > >
> > > > Julian
> > > >
> > > >
> > > > > On Sep 21, 2015, at 2:34 AM, Li Yang <li...@apache.org> wrote:
> > > > >
> > > > > @Julian, plan to refactor the underlying aggregation group in Q4.
> > Will
> > > > drop
> > > > > hierarchy concept in the backend, however in the frontend for ease
> of
> > > > > understanding, may still call it hierarchy.  :-)
> > > > >
> > > > > On Sat, Sep 19, 2015 at 2:40 AM, Patrick McAnneny <
> > > > > patrick.mcanneny@leadkarma.com> wrote:
> > > > >
> > > > >> Great thread and great read. I see now that hierarchies are just a
> > way
> > > > to
> > > > >> specify correlated dimensions for added efficiencies in cube
> > > > architecture.
> > > > >> I understand now. Thanks a lot!
> > > > >>
> > > > >> On Fri, Sep 18, 2015 at 1:08 PM, Julian Hyde <jh...@apache.org>
> > > wrote:
> > > > >>
> > > > >>> As I've said before[1] I don't think hierarchies add much to
> Kylin.
> > > > >>>
> > > > >>> Julian
> > > > >>>
> > > > >>> [1]
> > > > >>>
> > > > >>
> > > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/kylin-dev/201506.mbox/%3C487CDC6B-1B48-410E-B85D-981FA4F3E5B7@apache.org%3E
> > > > >>>
> > > > >>> On Fri, Sep 18, 2015 at 9:18 AM, Patrick McAnneny
> > > > >>> <pa...@leadkarma.com> wrote:
> > > > >>>> So does a regular dimension though right? What's the difference
> > > > >> between a
> > > > >>>> singular hierarchy dimension and a regular dimension? The end
> > > > resulting
> > > > >>>> kylin sql would be the same (group by) right? Thanks for helping
> > me
> > > > >>>> understand!
> > > > >>>>
> > > > >>>> On Fri, Sep 18, 2015 at 12:01 PM, Sarnath <st...@gmail.com>
> > > wrote:
> > > > >>>>
> > > > >>>>> Hierarchy indicates that drill down is possible.
> > > > >>>>>
> > > > >>>
> > > > >>
> > > >
> > > >
> > >
> >
>

Re: Hierarchy Dimension

[Patrick McAnneny <pa...@leadkarma.com>]

Agreed, but I'm saying more "what kylin does" rather than "what does this
term mean". For example, some trail and error is necessary for figuring out
various UI elements. But I'm sure all of this will improve over time and
the majority of the UI is great. Thanks!

On Mon, Sep 21, 2015 at 9:17 PM, Luke Han <lu...@gmail.com> wrote:

> As today's DW/BI world, Hierarchy Dimension is still very popular and
> common for BI experts and users, like most of traditional and modern
> products doing.
> I don't think it will confuse users especially when they design cubes.
>
> So keep the same concept is very important for user to adopt new stuff,
> here is Kylin, on new tech stack.
>
> Whatever backend setting will be, the user facing term, concept should
> always keep same with existing one as much as possible:)
>
> For documentation, there's terminology page would like to recommend
> newbie to go through first:
> http://kylin.incubator.apache.org/docs/gettingstarted/terminology.html
>
> There's some enhancement of current UI and tutorial documentation
> will come, but I don't think it make sense for a project to draft all basic
> and common concept and terms everywhere:-)
>
> Thanks.
>
>
> Best Regards!
> ---------------------
>
> Luke Han
>
> On Tue, Sep 22, 2015 at 4:15 AM, Patrick McAnneny <
> patrick.mcanneny@leadkarma.com> wrote:
>
> > I found it confusing initially until I read about how it is mostly a
> method
> > of optimising the cube (rather than a separate type of dimension that
> would
> > be queried differently). It makes a lot of sense now that I understand,
> but
> > perhaps a little bit more documentation would help others starting out.
> (it
> > is not covered in the cube construction slides)
> >
> > Not a huge deal though
> >
> > On Mon, Sep 21, 2015 at 3:21 PM, Julian Hyde <jh...@apache.org> wrote:
> >
> > > Yes, hierarchies may be a useful organizing concept. I claim that
> > > hierarchies are separate things: (1) navigation paths for a UI, (2)
> > > statistical patterns in data access. Conversations like this one are
> > > evidence that — at least for some people — hierarchies are confusing.
> So,
> > > keep them as a user concept only if they make life simpler for the
> > majority
> > > of users.
> > >
> > > Julian
> > >
> > >
> > > > On Sep 21, 2015, at 2:34 AM, Li Yang <li...@apache.org> wrote:
> > > >
> > > > @Julian, plan to refactor the underlying aggregation group in Q4.
> Will
> > > drop
> > > > hierarchy concept in the backend, however in the frontend for ease of
> > > > understanding, may still call it hierarchy.  :-)
> > > >
> > > > On Sat, Sep 19, 2015 at 2:40 AM, Patrick McAnneny <
> > > > patrick.mcanneny@leadkarma.com> wrote:
> > > >
> > > >> Great thread and great read. I see now that hierarchies are just a
> way
> > > to
> > > >> specify correlated dimensions for added efficiencies in cube
> > > architecture.
> > > >> I understand now. Thanks a lot!
> > > >>
> > > >> On Fri, Sep 18, 2015 at 1:08 PM, Julian Hyde <jh...@apache.org>
> > wrote:
> > > >>
> > > >>> As I've said before[1] I don't think hierarchies add much to Kylin.
> > > >>>
> > > >>> Julian
> > > >>>
> > > >>> [1]
> > > >>>
> > > >>
> > >
> >
> http://mail-archives.apache.org/mod_mbox/kylin-dev/201506.mbox/%3C487CDC6B-1B48-410E-B85D-981FA4F3E5B7@apache.org%3E
> > > >>>
> > > >>> On Fri, Sep 18, 2015 at 9:18 AM, Patrick McAnneny
> > > >>> <pa...@leadkarma.com> wrote:
> > > >>>> So does a regular dimension though right? What's the difference
> > > >> between a
> > > >>>> singular hierarchy dimension and a regular dimension? The end
> > > resulting
> > > >>>> kylin sql would be the same (group by) right? Thanks for helping
> me
> > > >>>> understand!
> > > >>>>
> > > >>>> On Fri, Sep 18, 2015 at 12:01 PM, Sarnath <st...@gmail.com>
> > wrote:
> > > >>>>
> > > >>>>> Hierarchy indicates that drill down is possible.
> > > >>>>>
> > > >>>
> > > >>
> > >
> > >
> >
>

Re: Hierarchy Dimension

[Luke Han <lu...@gmail.com>]

As today's DW/BI world, Hierarchy Dimension is still very popular and
common for BI experts and users, like most of traditional and modern
products doing.
I don't think it will confuse users especially when they design cubes.

So keep the same concept is very important for user to adopt new stuff,
here is Kylin, on new tech stack.

Whatever backend setting will be, the user facing term, concept should
always keep same with existing one as much as possible:)

For documentation, there's terminology page would like to recommend
newbie to go through first:
http://kylin.incubator.apache.org/docs/gettingstarted/terminology.html

There's some enhancement of current UI and tutorial documentation
will come, but I don't think it make sense for a project to draft all basic
and common concept and terms everywhere:-)

Thanks.


Best Regards!
---------------------

Luke Han

On Tue, Sep 22, 2015 at 4:15 AM, Patrick McAnneny <
patrick.mcanneny@leadkarma.com> wrote:

> I found it confusing initially until I read about how it is mostly a method
> of optimising the cube (rather than a separate type of dimension that would
> be queried differently). It makes a lot of sense now that I understand, but
> perhaps a little bit more documentation would help others starting out. (it
> is not covered in the cube construction slides)
>
> Not a huge deal though
>
> On Mon, Sep 21, 2015 at 3:21 PM, Julian Hyde <jh...@apache.org> wrote:
>
> > Yes, hierarchies may be a useful organizing concept. I claim that
> > hierarchies are separate things: (1) navigation paths for a UI, (2)
> > statistical patterns in data access. Conversations like this one are
> > evidence that — at least for some people — hierarchies are confusing. So,
> > keep them as a user concept only if they make life simpler for the
> majority
> > of users.
> >
> > Julian
> >
> >
> > > On Sep 21, 2015, at 2:34 AM, Li Yang <li...@apache.org> wrote:
> > >
> > > @Julian, plan to refactor the underlying aggregation group in Q4. Will
> > drop
> > > hierarchy concept in the backend, however in the frontend for ease of
> > > understanding, may still call it hierarchy.  :-)
> > >
> > > On Sat, Sep 19, 2015 at 2:40 AM, Patrick McAnneny <
> > > patrick.mcanneny@leadkarma.com> wrote:
> > >
> > >> Great thread and great read. I see now that hierarchies are just a way
> > to
> > >> specify correlated dimensions for added efficiencies in cube
> > architecture.
> > >> I understand now. Thanks a lot!
> > >>
> > >> On Fri, Sep 18, 2015 at 1:08 PM, Julian Hyde <jh...@apache.org>
> wrote:
> > >>
> > >>> As I've said before[1] I don't think hierarchies add much to Kylin.
> > >>>
> > >>> Julian
> > >>>
> > >>> [1]
> > >>>
> > >>
> >
> http://mail-archives.apache.org/mod_mbox/kylin-dev/201506.mbox/%3C487CDC6B-1B48-410E-B85D-981FA4F3E5B7@apache.org%3E
> > >>>
> > >>> On Fri, Sep 18, 2015 at 9:18 AM, Patrick McAnneny
> > >>> <pa...@leadkarma.com> wrote:
> > >>>> So does a regular dimension though right? What's the difference
> > >> between a
> > >>>> singular hierarchy dimension and a regular dimension? The end
> > resulting
> > >>>> kylin sql would be the same (group by) right? Thanks for helping me
> > >>>> understand!
> > >>>>
> > >>>> On Fri, Sep 18, 2015 at 12:01 PM, Sarnath <st...@gmail.com>
> wrote:
> > >>>>
> > >>>>> Hierarchy indicates that drill down is possible.
> > >>>>>
> > >>>
> > >>
> >
> >
>

Re: Hierarchy Dimension

[Patrick McAnneny <pa...@leadkarma.com>]

I found it confusing initially until I read about how it is mostly a method
of optimising the cube (rather than a separate type of dimension that would
be queried differently). It makes a lot of sense now that I understand, but
perhaps a little bit more documentation would help others starting out. (it
is not covered in the cube construction slides)

Not a huge deal though

On Mon, Sep 21, 2015 at 3:21 PM, Julian Hyde <jh...@apache.org> wrote:

> Yes, hierarchies may be a useful organizing concept. I claim that
> hierarchies are separate things: (1) navigation paths for a UI, (2)
> statistical patterns in data access. Conversations like this one are
> evidence that — at least for some people — hierarchies are confusing. So,
> keep them as a user concept only if they make life simpler for the majority
> of users.
>
> Julian
>
>
> > On Sep 21, 2015, at 2:34 AM, Li Yang <li...@apache.org> wrote:
> >
> > @Julian, plan to refactor the underlying aggregation group in Q4. Will
> drop
> > hierarchy concept in the backend, however in the frontend for ease of
> > understanding, may still call it hierarchy.  :-)
> >
> > On Sat, Sep 19, 2015 at 2:40 AM, Patrick McAnneny <
> > patrick.mcanneny@leadkarma.com> wrote:
> >
> >> Great thread and great read. I see now that hierarchies are just a way
> to
> >> specify correlated dimensions for added efficiencies in cube
> architecture.
> >> I understand now. Thanks a lot!
> >>
> >> On Fri, Sep 18, 2015 at 1:08 PM, Julian Hyde <jh...@apache.org> wrote:
> >>
> >>> As I've said before[1] I don't think hierarchies add much to Kylin.
> >>>
> >>> Julian
> >>>
> >>> [1]
> >>>
> >>
> http://mail-archives.apache.org/mod_mbox/kylin-dev/201506.mbox/%3C487CDC6B-1B48-410E-B85D-981FA4F3E5B7@apache.org%3E
> >>>
> >>> On Fri, Sep 18, 2015 at 9:18 AM, Patrick McAnneny
> >>> <pa...@leadkarma.com> wrote:
> >>>> So does a regular dimension though right? What's the difference
> >> between a
> >>>> singular hierarchy dimension and a regular dimension? The end
> resulting
> >>>> kylin sql would be the same (group by) right? Thanks for helping me
> >>>> understand!
> >>>>
> >>>> On Fri, Sep 18, 2015 at 12:01 PM, Sarnath <st...@gmail.com> wrote:
> >>>>
> >>>>> Hierarchy indicates that drill down is possible.
> >>>>>
> >>>
> >>
>
>

Re: Hierarchy Dimension

[Julian Hyde <jh...@apache.org>]

Yes, hierarchies may be a useful organizing concept. I claim that hierarchies are separate things: (1) navigation paths for a UI, (2) statistical patterns in data access. Conversations like this one are evidence that — at least for some people — hierarchies are confusing. So, keep them as a user concept only if they make life simpler for the majority of users.

Julian


> On Sep 21, 2015, at 2:34 AM, Li Yang <li...@apache.org> wrote:
> 
> @Julian, plan to refactor the underlying aggregation group in Q4. Will drop
> hierarchy concept in the backend, however in the frontend for ease of
> understanding, may still call it hierarchy.  :-)
> 
> On Sat, Sep 19, 2015 at 2:40 AM, Patrick McAnneny <
> patrick.mcanneny@leadkarma.com> wrote:
> 
>> Great thread and great read. I see now that hierarchies are just a way to
>> specify correlated dimensions for added efficiencies in cube architecture.
>> I understand now. Thanks a lot!
>> 
>> On Fri, Sep 18, 2015 at 1:08 PM, Julian Hyde <jh...@apache.org> wrote:
>> 
>>> As I've said before[1] I don't think hierarchies add much to Kylin.
>>> 
>>> Julian
>>> 
>>> [1]
>>> 
>> http://mail-archives.apache.org/mod_mbox/kylin-dev/201506.mbox/%3C487CDC6B-1B48-410E-B85D-981FA4F3E5B7@apache.org%3E
>>> 
>>> On Fri, Sep 18, 2015 at 9:18 AM, Patrick McAnneny
>>> <pa...@leadkarma.com> wrote:
>>>> So does a regular dimension though right? What's the difference
>> between a
>>>> singular hierarchy dimension and a regular dimension? The end resulting
>>>> kylin sql would be the same (group by) right? Thanks for helping me
>>>> understand!
>>>> 
>>>> On Fri, Sep 18, 2015 at 12:01 PM, Sarnath <st...@gmail.com> wrote:
>>>> 
>>>>> Hierarchy indicates that drill down is possible.
>>>>> 
>>> 
>> 

Re: Hierarchy Dimension

[Li Yang <li...@apache.org>]

@Julian, plan to refactor the underlying aggregation group in Q4. Will drop
hierarchy concept in the backend, however in the frontend for ease of
understanding, may still call it hierarchy.  :-)

On Sat, Sep 19, 2015 at 2:40 AM, Patrick McAnneny <
patrick.mcanneny@leadkarma.com> wrote:

> Great thread and great read. I see now that hierarchies are just a way to
> specify correlated dimensions for added efficiencies in cube architecture.
> I understand now. Thanks a lot!
>
> On Fri, Sep 18, 2015 at 1:08 PM, Julian Hyde <jh...@apache.org> wrote:
>
> > As I've said before[1] I don't think hierarchies add much to Kylin.
> >
> > Julian
> >
> > [1]
> >
> http://mail-archives.apache.org/mod_mbox/kylin-dev/201506.mbox/%3C487CDC6B-1B48-410E-B85D-981FA4F3E5B7@apache.org%3E
> >
> > On Fri, Sep 18, 2015 at 9:18 AM, Patrick McAnneny
> > <pa...@leadkarma.com> wrote:
> > > So does a regular dimension though right? What's the difference
> between a
> > > singular hierarchy dimension and a regular dimension? The end resulting
> > > kylin sql would be the same (group by) right? Thanks for helping me
> > > understand!
> > >
> > > On Fri, Sep 18, 2015 at 12:01 PM, Sarnath <st...@gmail.com> wrote:
> > >
> > >> Hierarchy indicates that drill down is possible.
> > >>
> >
>

Re: Hierarchy Dimension

[Patrick McAnneny <pa...@leadkarma.com>]

Great thread and great read. I see now that hierarchies are just a way to
specify correlated dimensions for added efficiencies in cube architecture.
I understand now. Thanks a lot!

On Fri, Sep 18, 2015 at 1:08 PM, Julian Hyde <jh...@apache.org> wrote:

> As I've said before[1] I don't think hierarchies add much to Kylin.
>
> Julian
>
> [1]
> http://mail-archives.apache.org/mod_mbox/kylin-dev/201506.mbox/%3C487CDC6B-1B48-410E-B85D-981FA4F3E5B7@apache.org%3E
>
> On Fri, Sep 18, 2015 at 9:18 AM, Patrick McAnneny
> <pa...@leadkarma.com> wrote:
> > So does a regular dimension though right? What's the difference between a
> > singular hierarchy dimension and a regular dimension? The end resulting
> > kylin sql would be the same (group by) right? Thanks for helping me
> > understand!
> >
> > On Fri, Sep 18, 2015 at 12:01 PM, Sarnath <st...@gmail.com> wrote:
> >
> >> Hierarchy indicates that drill down is possible.
> >>
>

Re: Hierarchy Dimension

[Julian Hyde <jh...@apache.org>]

As I've said before[1] I don't think hierarchies add much to Kylin.

Julian

[1] http://mail-archives.apache.org/mod_mbox/kylin-dev/201506.mbox/%3C487CDC6B-1B48-410E-B85D-981FA4F3E5B7@apache.org%3E

On Fri, Sep 18, 2015 at 9:18 AM, Patrick McAnneny
<pa...@leadkarma.com> wrote:
> So does a regular dimension though right? What's the difference between a
> singular hierarchy dimension and a regular dimension? The end resulting
> kylin sql would be the same (group by) right? Thanks for helping me
> understand!
>
> On Fri, Sep 18, 2015 at 12:01 PM, Sarnath <st...@gmail.com> wrote:
>
>> Hierarchy indicates that drill down is possible.
>>

Re: Hierarchy Dimension

[Patrick McAnneny <pa...@leadkarma.com>]

So does a regular dimension though right? What's the difference between a
singular hierarchy dimension and a regular dimension? The end resulting
kylin sql would be the same (group by) right? Thanks for helping me
understand!

On Fri, Sep 18, 2015 at 12:01 PM, Sarnath <st...@gmail.com> wrote:

> Hierarchy indicates that drill down is possible.
>

Re: Hierarchy Dimension

[Sarnath <st...@gmail.com>]

Hierarchy indicates that drill down is possible.

Re: Hierarchy Dimension

[Patrick McAnneny <pa...@leadkarma.com>]

I see. How would this be different than a normal group by though when
running queries in Kylin sql? What is the difference between specifying
this hierarchy and just a normal dimension for each?

On Fri, Sep 18, 2015 at 3:22 AM, Li Yang <li...@apache.org> wrote:

> Yes, the "roll-up" understanding is correct.
>
> Defining one hierarchy made up of the three dimensions is good enough. It
> covers all 3 roll-ups. The order of dimensions matters because hierarchy
> basically describes a "contains" relationship, like CONTINENT => COUNTRY =>
> CITY. A wrong order will impact performance at both build time and query
> time.
>
> On Thu, Sep 17, 2015 at 11:27 PM, Patrick McAnneny <
> patrick.mcanneny@leadkarma.com> wrote:
>
> > The way I understand the hierarchy feature (lets say with three columns
> > specified) is the following: "create a roll-up aggregate grouping by this
> > column, then that column, then that column" Is this correct
> understanding?
> >
> >  If I would want to query potentially based on any combination of those
> > three roll-ups, would it be best to create three separate hierarchy
> > dimensions or the combination of each? Would order even matter here?
> Order
> > does not usually matter in 'group by' statements
> >
>

Re: Hierarchy Dimension

[Li Yang <li...@apache.org>]

Yes, the "roll-up" understanding is correct.

Defining one hierarchy made up of the three dimensions is good enough. It
covers all 3 roll-ups. The order of dimensions matters because hierarchy
basically describes a "contains" relationship, like CONTINENT => COUNTRY =>
CITY. A wrong order will impact performance at both build time and query
time.

On Thu, Sep 17, 2015 at 11:27 PM, Patrick McAnneny <
patrick.mcanneny@leadkarma.com> wrote:

> The way I understand the hierarchy feature (lets say with three columns
> specified) is the following: "create a roll-up aggregate grouping by this
> column, then that column, then that column" Is this correct understanding?
>
>  If I would want to query potentially based on any combination of those
> three roll-ups, would it be best to create three separate hierarchy
> dimensions or the combination of each? Would order even matter here? Order
> does not usually matter in 'group by' statements
>