You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by 251469031 <25...@qq.com> on 2016/06/15 01:32:17 UTC
some doubt about hierarchy dimension in kylin 1.5+
Hi all:
There are 2 questions bothering me as follows:
1. In kylin 1.5+, the type of dimensions includes 'Normal' & 'Derived', and 'Hierarchy' is removed from the dimension tab button to aggregation groups, being a kind of select_rule(@see New Aggregation Group). For the hierarchy dimension, what kind of dimensions should I choose when designing a cube, normal or derived?
2. In API kylin/api/cube_desc/{cubeName}, I can get derived columns' dimension name, e.g. In the examples below, the dimension name of derived column DIMB_CODE is '维度A', but for the hierarchy columns, e.g. If I have a hierarchy dimension named date, and the levels are year, month, day. In the API I can only figure out year, month and day is three level of a dimension, but I don't know which dimension are they belong to?
The following json string is an example of my test case. Looking forward to your help, thanks~~
e.g.
{
"uuid":"b4307b44-8ca7-410d-ab29-5ac7974f77e7",
"version":"1.5.2",
"name":"C_Customer_SaveB",
"description":"客户资产分析模型",
"dimensions":[
{
"name":"维度A",
"table":"DEFAULT.DIMA_DI_D",
"column":null,
"derived":[
"DIMA_CODE",
"DIMA_NAME"
]
},
{
"name":"维度B",
"table":"DEFAULT.DIMB_DI_D",
"column":null,
"derived":[
"DIMB_CODE",
"DIMB_NAME"
]
},
{
"name":"维度C_L1",
"table":"DEFAULT.DIMC_DI_H",
"column":"DIMC_LEVEL_1",
"derived":null
},
{
"name":"维度C_L2",
"table":"DEFAULT.DIMC_DI_H",
"column":"DIMC_LEVEL_2",
"derived":null
},
{
"name":"维度C_L3",
"table":"DEFAULT.DIMC_DI_H",
"column":"DIMC_LEVEL_3",
"derived":null
},
{
"name":"维度D_L1",
"table":"DEFAULT.DIMD_DI_H",
"column":"DIMD_LEVEL_1",
"derived":null
},
{
"name":"维度D_L2",
"table":"DEFAULT.DIMD_DI_H",
"column":"DIMD_LEVEL_2",
"derived":null
},
{
"name":"维度D_L3",
"table":"DEFAULT.DIMD_DI_H",
"column":"DIMD_LEVEL_3",
"derived":null
}
],
"measures":[
{
"name":"_COUNT_",
"function":{
"expression":"COUNT",
"parameter":{
"type":"constant",
"value":"1",
"next_parameter":null
},
"returntype":"bigint"
},
"dependent_measure_ref":null
},
{
"name":"度量SUM",
"function":{
"expression":"SUM",
"parameter":{
"type":"column",
"value":"MEASURE",
"next_parameter":null
},
"returntype":"decimal"
},
"dependent_measure_ref":null
},
{
"name":"度量MIN",
"function":{
"expression":"MIN",
"parameter":{
"type":"column",
"value":"MEASURE",
"next_parameter":null
},
"returntype":"double"
},
"dependent_measure_ref":null
},
{
"name":"度量MAX",
"function":{
"expression":"MAX",
"parameter":{
"type":"column",
"value":"MEASURE",
"next_parameter":null
},
"returntype":"double"
},
"dependent_measure_ref":null
},
{
"name":"度量COUNT_DISTINCT",
"function":{
"expression":"COUNT_DISTINCT",
"parameter":{
"type":"column",
"value":"MEASURE",
"next_parameter":null
},
"returntype":"hllc10"
},
"dependent_measure_ref":null
},
{
"name":"度量TOPN",
"function":{
"expression":"TOP_N",
"parameter":{
"type":"column",
"value":"MEASURE",
"next_parameter":{
"type":"column",
"value":"DIMA_CODE",
"next_parameter":null
}
},
"returntype":"topn(1000)"
},
"dependent_measure_ref":null
},
{
"name":"度量_RAW",
"function":{
"expression":"RAW",
"parameter":{
"type":"column",
"value":"MEASURE",
"next_parameter":null
},
"returntype":"raw"
},
"dependent_measure_ref":null
}
],
"rowkey":{
"rowkey_columns":[
{
"column":"DIMA_ID",
"encoding":"dict",
"isShardBy":false
},
{
"column":"DIMB_ID",
"encoding":"dict",
"isShardBy":false
},
{
"column":"DIMC_LEVEL_1",
"encoding":"dict",
"isShardBy":false
},
{
"column":"DIMC_LEVEL_2",
"encoding":"dict",
"isShardBy":false
},
{
"column":"DIMC_LEVEL_3",
"encoding":"dict",
"isShardBy":false
},
{
"column":"DIMD_LEVEL_1",
"encoding":"dict",
"isShardBy":false
},
{
"column":"DIMD_LEVEL_2",
"encoding":"dict",
"isShardBy":false
},
{
"column":"DIMD_LEVEL_3",
"encoding":"dict",
"isShardBy":false
}
]
},
"signature":"NMpOd+FGWvuTwP+neqeMQg==",
"last_modified":1465787014776,
"model_name":"M_Customer_SaveB",
"null_string":null,
"hbase_mapping":{
"column_family":[
{
"name":"F1",
"columns":[
{
"qualifier":"M",
"measure_refs":[
"_COUNT_",
"度量SUM",
"度量MIN",
"度量MAX",
"度量TOPN",
"度量_RAW"
]
}
]
},
{
"name":"F2",
"columns":[
{
"qualifier":"M",
"measure_refs":[
"度量COUNT_DISTINCT"
]
}
]
}
]
},
"aggregation_groups":[
{
"includes":[
"DIMA_ID",
"DIMB_ID",
"DIMC_LEVEL_1",
"DIMC_LEVEL_2",
"DIMC_LEVEL_3",
"DIMD_LEVEL_1",
"DIMD_LEVEL_2",
"DIMD_LEVEL_3"
],
"select_rule":{
"hierarchy_dims":[
[
"DIMC_LEVEL_1",
"DIMC_LEVEL_2",
"DIMC_LEVEL_3"
],
[
"DIMD_LEVEL_1",
"DIMD_LEVEL_2",
"DIMD_LEVEL_3"
]
],
"mandatory_dims":[
"DIMA_ID"
],
"joint_dims":[
]
}
}
],
"notify_list":[
"251469031@qq.com",
"liyinwei@cvte.com"
],
"status_need_notify":[
"ERROR",
"DISCARDED",
"SUCCEED"
],
"partition_date_start":1465689600000,
"partition_date_end":3153600000000,
"auto_merge_time_ranges":[
604800000,
2419200000
],
"retention_range":0,
"engine_type":2,
"storage_type":2,
"override_kylin_properties":{
}
}
Re: some doubt about hierarchy dimension in kylin 1.5+
Posted by hongbin ma <ma...@apache.org>.
currently there's no plan for such requirements
On Thu, Jun 16, 2016 at 9:54 AM, 李寅威 <25...@qq.com> wrote:
> yep, that's what I mean.
>
>
> so would kylin api consider this scene in the future, or this requirement
> itself is not reasonable?
>
>
> ------------------
> 李寅威 | CVTE
> liyinwei@cvte.com
> 广州视源电子科技股份有限公司
> Guangzhou Shiyuan Electronics Co., Ltd.
>
>
>
>
>
>
>
> ------------------ 原始邮件 ------------------
> 发件人: "mahongbin";<ma...@apache.org>;
> 发送时间: 2016年6月16日(星期四) 上午9:49
> 收件人: "李寅威"<25...@qq.com>;
> 抄送: "dev"<de...@kylin.apache.org>;
> 主题: Re: some doubt about hierarchy dimension in kylin 1.5+
>
>
>
> date dimension does not exist at all, it's just a logical "dimension"
> consists of three related dimensions: year,month,day
>
> how can you get "date" in the json when you didn't input "date" while
> creating cube?
>
> On Thu, Jun 16, 2016 at 8:44 AM, 251469031 <25...@qq.com> wrote:
>
> > @Hongbin Ma
> >
> > sorry about that.
> >
> > the second question, e.g. I have a dimension named date, and its levels
> > are year, month and day, I want to display the hierarchy with
> > indentation to the endusers like this:
> >
> > date
> > --year
> > --month
> > --day
> >
> > The problem is, I can't get the name "date" from kylin API. Looking
> > forward to your help, thx~
> >
> >
> > ------------------ 原始邮件 ------------------
> > *发件人:* "hongbin ma";<ma...@apache.org>;
> > *发送时间:* 2016年6月15日(星期三) 晚上10:14
> > *收件人:* "dev"<de...@kylin.apache.org>;
> > *主题:* Re: some doubt about hierarchy dimension in kylin 1.5+
> >
> > 1. use normal dimension here. the hierachy relationship is later defined
> in
> > "aggregation group"
> > 2. actually I don't quite understand your question. if you're asking how
> to
> > get the hierachy relationship from the json, take a look
> > at aggregation_groups => select_rule => hierarchy_dims
> >
> >
> > --
> > Regards,
> >
> > *Bin Mahone | 马洪宾*
> > Apache Kylin: http://kylin.io
> > Github: https://github.com/binmahone
> >
> >
>
>
> --
> Regards,
>
> *Bin Mahone | 马洪宾*
> Apache Kylin: http://kylin.io
> Github: https://github.com/binmahone
>
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone
回复： some doubt about hierarchy dimension in kylin 1.5+
Posted by 李寅威 <25...@qq.com>.
yep, that's what I mean.
so would kylin api consider this scene in the future, or this requirement itself is not reasonable?
------------------
李寅威 | CVTE
liyinwei@cvte.com
广州视源电子科技股份有限公司
Guangzhou Shiyuan Electronics Co., Ltd.
------------------ 原始邮件 ------------------
发件人: "mahongbin";<ma...@apache.org>;
发送时间: 2016年6月16日(星期四) 上午9:49
收件人: "李寅威"<25...@qq.com>;
抄送: "dev"<de...@kylin.apache.org>;
主题: Re: some doubt about hierarchy dimension in kylin 1.5+
date dimension does not exist at all, it's just a logical "dimension"
consists of three related dimensions: year,month,day
how can you get "date" in the json when you didn't input "date" while
creating cube?
On Thu, Jun 16, 2016 at 8:44 AM, 251469031 <25...@qq.com> wrote:
> @Hongbin Ma
>
> sorry about that.
>
> the second question, e.g. I have a dimension named date, and its levels
> are year, month and day, I want to display the hierarchy with
> indentation to the endusers like this:
>
> date
> --year
> --month
> --day
>
> The problem is, I can't get the name "date" from kylin API. Looking
> forward to your help, thx~
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "hongbin ma";<ma...@apache.org>;
> *发送时间:* 2016年6月15日(星期三) 晚上10:14
> *收件人:* "dev"<de...@kylin.apache.org>;
> *主题:* Re: some doubt about hierarchy dimension in kylin 1.5+
>
> 1. use normal dimension here. the hierachy relationship is later defined in
> "aggregation group"
> 2. actually I don't quite understand your question. if you're asking how to
> get the hierachy relationship from the json, take a look
> at aggregation_groups => select_rule => hierarchy_dims
>
>
> --
> Regards,
>
> *Bin Mahone | 马洪宾*
> Apache Kylin: http://kylin.io
> Github: https://github.com/binmahone
>
>
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone
Re: some doubt about hierarchy dimension in kylin 1.5+
Posted by hongbin ma <ma...@apache.org>.
date dimension does not exist at all, it's just a logical "dimension"
consists of three related dimensions: year,month,day
how can you get "date" in the json when you didn't input "date" while
creating cube?
On Thu, Jun 16, 2016 at 8:44 AM, 251469031 <25...@qq.com> wrote:
> @Hongbin Ma
>
> sorry about that.
>
> the second question, e.g. I have a dimension named date, and its levels
> are year, month and day, I want to display the hierarchy with
> indentation to the endusers like this:
>
> date
> --year
> --month
> --day
>
> The problem is, I can't get the name "date" from kylin API. Looking
> forward to your help, thx~
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "hongbin ma";<ma...@apache.org>;
> *发送时间:* 2016年6月15日(星期三) 晚上10:14
> *收件人:* "dev"<de...@kylin.apache.org>;
> *主题:* Re: some doubt about hierarchy dimension in kylin 1.5+
>
> 1. use normal dimension here. the hierachy relationship is later defined in
> "aggregation group"
> 2. actually I don't quite understand your question. if you're asking how to
> get the hierachy relationship from the json, take a look
> at aggregation_groups => select_rule => hierarchy_dims
>
>
> --
> Regards,
>
> *Bin Mahone | 马洪宾*
> Apache Kylin: http://kylin.io
> Github: https://github.com/binmahone
>
>
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone
回复： some doubt about hierarchy dimension in kylin 1.5+
Posted by 251469031 <25...@qq.com>.
@Hongbin Ma
sorry about that.
the second question, e.g. I have a dimension named date, and its levels are year, month and day, I want to display the hierarchy with indentation to the endusers like this:
date
--year
--month
--day
The problem is, I can't get the name "date" from kylin API. Looking forward to your help, thx~
------------------ 原始邮件 ------------------
发件人: "hongbin ma";<ma...@apache.org>;
发送时间: 2016年6月15日(星期三) 晚上10:14
收件人: "dev"<de...@kylin.apache.org>;
主题: Re: some doubt about hierarchy dimension in kylin 1.5+
1. use normal dimension here. the hierachy relationship is later defined in
"aggregation group"
2. actually I don't quite understand your question. if you're asking how to
get the hierachy relationship from the json, take a look
at aggregation_groups => select_rule => hierarchy_dims
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone
Re: some doubt about hierarchy dimension in kylin 1.5+
Posted by hongbin ma <ma...@apache.org>.
1. use normal dimension here. the hierachy relationship is later defined in
"aggregation group"
2. actually I don't quite understand your question. if you're asking how to
get the hierachy relationship from the json, take a look
at aggregation_groups => select_rule => hierarchy_dims
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone