You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by ShaoFeng Shi <sh...@apache.org> on 2017/11/27 05:40:23 UTC
Re: [Discuss] Disable/hide "RAW" measure in Kylin web GUI
Last year I raised this discussion but didn't have follow-up action.
Now we see there are still some new users misuse this feature, and then
face performance and maintenance issues.
In Kylin 2.1, the new "query pushdown" feature can forward the Cube
unmatched queries to alternative query engines like Hive / SparkSQL. The
raw data query is just such a scenario.
So I think it is time to disable the RAW measure on Kylin now. JIRA
created for it: https://issues.apache.org/jira/browse/KYLIN-3062
Please comment if you see any issue.
2016-12-19 22:03 GMT+08:00 Billy Liu <bi...@apache.org>:
> The experimental mode is system wide feature toggle. I think case by case
> is more flexible. Most new features could have toggles, default are off.
>
> 2016-12-19 21:40 GMT+08:00 Luke Han <lu...@gmail.com>:
>
> > Beta or Experimental will also bring confusing for most of users.
> >
> > Maybe we could have something called "expert" or "experimental" model in
> > system configuration.
> >
> > User will not see such content since they will be hidden by default but
> > admin could set it to true if they confident to enable such features.
> >
> > How do you think?
> >
> >
> > Best Regards!
> > ---------------------
> >
> > Luke Han
> >
> > On Mon, Dec 19, 2016 at 11:24 AM, Xiaoyu Wang <wa...@gmail.com>
> wrote:
> >
> > > I’m sorry for too long not maintain it!
> > >
> > > I agree with liyang to give a label to the RAW measure “Beta” or
> others.
> > >
> > > I will improve it when I have time!
> > >
> > > 2016-12-19 10:21 GMT+08:00 Li Yang <li...@apache.org>:
> > >
> > > > Or display the Raw with "Beta" or "Experimental" to warn user that it
> > is
> > > > not a mature feature?
> > > >
> > > > On Fri, Dec 16, 2016 at 12:30 PM, 康凯森 <ka...@qq.com> wrote:
> > > >
> > > > > +1.
> > > > > But the "RAW" measure is still some useful, we could improve it
> next
> > > year
> > > > > when we have time.
> > > > >
> > > > >
> > > > > ------------------ 原始邮件 ------------------
> > > > > 发件人: "ShaoFeng Shi";<sh...@apache.org>;
> > > > > 发送时间: 2016年12月15日(星期四) 中午12:05
> > > > > 收件人: "dev"<de...@kylin.apache.org>;
> > > > >
> > > > > 主题: [Discuss] Disable/hide "RAW" measure in Kylin web GUI
> > > > >
> > > > >
> > > > >
> > > > > Hello developers and users,
> > > > >
> > > > > I have a proposal want to disucss here, which is about the "RAW"
> > > measure
> > > > in
> > > > > Kylin.
> > > > >
> > > > > The RAW measure was developed to solve the requirement for getting
> > raw
> > > > data
> > > > > when user drill down from high level to low levels. It's
> performance
> > > > would
> > > > > be much better than fetching from source like Hive, so some users
> > like
> > > > it.
> > > > > This blog introduces it:
> > > > > https://kylin.apache.org/blog/2016/05/29/raw-measure-in-kylin/
> > > > >
> > > > > While it has some limitations:
> > > > > 1) always use dictionary encoding, which means it couldn't support
> > UHC;
> > > > > since raw columns usually be transaction IDs, numbers, etc, the
> > > building
> > > > > cost is much higher than ordinary dimensions;
> > > > > 2) the raw messages for a dimension combination are persisted in
> one
> > > big
> > > > > cell,; When too many rows dumped in one cell, will get
> BufferOverflow
> > > > > error. This couldn't be predicted, so for an modeler or analyst he
> > > > doesn't
> > > > > know whether this feature could work when he creates the cube.
> > > > > 3) seems no people maintains it;
> > > > >
> > > > > Based on above, I propose hiding this measure on Web GUI by
> default,
> > so
> > > > > avoiding confusing user; If someone hear and want to use this, he
> > still
> > > > can
> > > > > enable it by simply set in kylin.properties.
> > > > >
> > > > > Any input is welcomed.
> > > > >
> > > > > --
> > > > > Best regards,
> > > > >
> > > > > Shaofeng Shi 史少锋
> > > > >
> > > >
> > >
> >
>
--
Best regards,
Shaofeng Shi 史少锋
Re: [Discuss] Disable/hide "RAW" measure in Kylin web GUI
Posted by Li Yang <li...@apache.org>.
+1 for complete removal.
On Wed, Nov 29, 2017 at 1:45 PM, ShaoFeng Shi <sh...@apache.org>
wrote:
> Billy, I don't see a need for "advanced user could enable it also." Do
> you have such a case?
>
> For the legacy cube, it can continue to build, the backend code won't be
> removed during this phase.
>
>
> 2017-11-28 22:58 GMT+08:00 Billy Liu <bi...@apache.org>:
>
>> +1 to turn this feature off by default. The advanced user could enable it
>> also.
>>
>> 2017-11-27 13:53 GMT+08:00 Luke Han <lu...@gmail.com>:
>>
>>> +1 to remove it from new release, people could backport to new version
>>> with previous code
>>>
>>>
>>> Best Regards!
>>> ---------------------
>>>
>>> Luke Han
>>>
>>> On Sun, Nov 26, 2017 at 9:40 PM, ShaoFeng Shi <sh...@apache.org>
>>> wrote:
>>>
>>>> Last year I raised this discussion but didn't have follow-up action.
>>>>
>>>> Now we see there are still some new users misuse this feature, and then
>>>> face performance and maintenance issues.
>>>>
>>>> In Kylin 2.1, the new "query pushdown" feature can forward the Cube
>>>> unmatched queries to alternative query engines like Hive / SparkSQL. The
>>>> raw data query is just such a scenario.
>>>>
>>>> So I think it is time to disable the RAW measure on Kylin now. JIRA
>>>> created for it: https://issues.apache.org/jira/browse/KYLIN-3062
>>>>
>>>> Please comment if you see any issue.
>>>>
>>>> 2016-12-19 22:03 GMT+08:00 Billy Liu <bi...@apache.org>:
>>>>
>>>> > The experimental mode is system wide feature toggle. I think case by
>>>> case
>>>> > is more flexible. Most new features could have toggles, default are
>>>> off.
>>>> >
>>>> > 2016-12-19 21:40 GMT+08:00 Luke Han <lu...@gmail.com>:
>>>> >
>>>> > > Beta or Experimental will also bring confusing for most of users.
>>>> > >
>>>> > > Maybe we could have something called "expert" or "experimental"
>>>> model in
>>>> > > system configuration.
>>>> > >
>>>> > > User will not see such content since they will be hidden by default
>>>> but
>>>> > > admin could set it to true if they confident to enable such
>>>> features.
>>>> > >
>>>> > > How do you think?
>>>> > >
>>>> > >
>>>> > > Best Regards!
>>>> > > ---------------------
>>>> > >
>>>> > > Luke Han
>>>> > >
>>>> > > On Mon, Dec 19, 2016 at 11:24 AM, Xiaoyu Wang <wa...@gmail.com>
>>>> > wrote:
>>>> > >
>>>> > > > I’m sorry for too long not maintain it!
>>>> > > >
>>>> > > > I agree with liyang to give a label to the RAW measure “Beta” or
>>>> > others.
>>>> > > >
>>>> > > > I will improve it when I have time!
>>>> > > >
>>>> > > > 2016-12-19 10:21 GMT+08:00 Li Yang <li...@apache.org>:
>>>> > > >
>>>> > > > > Or display the Raw with "Beta" or "Experimental" to warn user
>>>> that it
>>>> > > is
>>>> > > > > not a mature feature?
>>>> > > > >
>>>> > > > > On Fri, Dec 16, 2016 at 12:30 PM, 康凯森 <ka...@qq.com>
>>>> wrote:
>>>> > > > >
>>>> > > > > > +1.
>>>> > > > > > But the "RAW" measure is still some useful, we could improve
>>>> it
>>>> > next
>>>> > > > year
>>>> > > > > > when we have time.
>>>> > > > > >
>>>> > > > > >
>>>> > > > > > ------------------ 原始邮件 ------------------
>>>> > > > > > 发件人: "ShaoFeng Shi";<sh...@apache.org>;
>>>> > > > > > 发送时间: 2016年12月15日(星期四) 中午12:05
>>>> > > > > > 收件人: "dev"<de...@kylin.apache.org>;
>>>> > > > > >
>>>> > > > > > 主题: [Discuss] Disable/hide "RAW" measure in Kylin web GUI
>>>> > > > > >
>>>> > > > > >
>>>> > > > > >
>>>> > > > > > Hello developers and users,
>>>> > > > > >
>>>> > > > > > I have a proposal want to disucss here, which is about the
>>>> "RAW"
>>>> > > > measure
>>>> > > > > in
>>>> > > > > > Kylin.
>>>> > > > > >
>>>> > > > > > The RAW measure was developed to solve the requirement for
>>>> getting
>>>> > > raw
>>>> > > > > data
>>>> > > > > > when user drill down from high level to low levels. It's
>>>> > performance
>>>> > > > > would
>>>> > > > > > be much better than fetching from source like Hive, so some
>>>> users
>>>> > > like
>>>> > > > > it.
>>>> > > > > > This blog introduces it:
>>>> > > > > > https://kylin.apache.org/blog/2016/05/29/raw-measure-in-kyli
>>>> n/
>>>> > > > > >
>>>> > > > > > While it has some limitations:
>>>> > > > > > 1) always use dictionary encoding, which means it couldn't
>>>> support
>>>> > > UHC;
>>>> > > > > > since raw columns usually be transaction IDs, numbers, etc,
>>>> the
>>>> > > > building
>>>> > > > > > cost is much higher than ordinary dimensions;
>>>> > > > > > 2) the raw messages for a dimension combination are persisted
>>>> in
>>>> > one
>>>> > > > big
>>>> > > > > > cell,; When too many rows dumped in one cell, will get
>>>> > BufferOverflow
>>>> > > > > > error. This couldn't be predicted, so for an modeler or
>>>> analyst he
>>>> > > > > doesn't
>>>> > > > > > know whether this feature could work when he creates the cube.
>>>> > > > > > 3) seems no people maintains it;
>>>> > > > > >
>>>> > > > > > Based on above, I propose hiding this measure on Web GUI by
>>>> > default,
>>>> > > so
>>>> > > > > > avoiding confusing user; If someone hear and want to use
>>>> this, he
>>>> > > still
>>>> > > > > can
>>>> > > > > > enable it by simply set in kylin.properties.
>>>> > > > > >
>>>> > > > > > Any input is welcomed.
>>>> > > > > >
>>>> > > > > > --
>>>> > > > > > Best regards,
>>>> > > > > >
>>>> > > > > > Shaofeng Shi 史少锋
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>>
>>>> Shaofeng Shi 史少锋
>>>>
>>>
>>>
>>
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
>
Re: [Discuss] Disable/hide "RAW" measure in Kylin web GUI
Posted by ShaoFeng Shi <sh...@apache.org>.
Billy, I don't see a need for "advanced user could enable it also." Do you
have such a case?
For the legacy cube, it can continue to build, the backend code won't be
removed during this phase.
2017-11-28 22:58 GMT+08:00 Billy Liu <bi...@apache.org>:
> +1 to turn this feature off by default. The advanced user could enable it
> also.
>
> 2017-11-27 13:53 GMT+08:00 Luke Han <lu...@gmail.com>:
>
>> +1 to remove it from new release, people could backport to new version
>> with previous code
>>
>>
>> Best Regards!
>> ---------------------
>>
>> Luke Han
>>
>> On Sun, Nov 26, 2017 at 9:40 PM, ShaoFeng Shi <sh...@apache.org>
>> wrote:
>>
>>> Last year I raised this discussion but didn't have follow-up action.
>>>
>>> Now we see there are still some new users misuse this feature, and then
>>> face performance and maintenance issues.
>>>
>>> In Kylin 2.1, the new "query pushdown" feature can forward the Cube
>>> unmatched queries to alternative query engines like Hive / SparkSQL. The
>>> raw data query is just such a scenario.
>>>
>>> So I think it is time to disable the RAW measure on Kylin now. JIRA
>>> created for it: https://issues.apache.org/jira/browse/KYLIN-3062
>>>
>>> Please comment if you see any issue.
>>>
>>> 2016-12-19 22:03 GMT+08:00 Billy Liu <bi...@apache.org>:
>>>
>>> > The experimental mode is system wide feature toggle. I think case by
>>> case
>>> > is more flexible. Most new features could have toggles, default are
>>> off.
>>> >
>>> > 2016-12-19 21:40 GMT+08:00 Luke Han <lu...@gmail.com>:
>>> >
>>> > > Beta or Experimental will also bring confusing for most of users.
>>> > >
>>> > > Maybe we could have something called "expert" or "experimental"
>>> model in
>>> > > system configuration.
>>> > >
>>> > > User will not see such content since they will be hidden by default
>>> but
>>> > > admin could set it to true if they confident to enable such features.
>>> > >
>>> > > How do you think?
>>> > >
>>> > >
>>> > > Best Regards!
>>> > > ---------------------
>>> > >
>>> > > Luke Han
>>> > >
>>> > > On Mon, Dec 19, 2016 at 11:24 AM, Xiaoyu Wang <wa...@gmail.com>
>>> > wrote:
>>> > >
>>> > > > I’m sorry for too long not maintain it!
>>> > > >
>>> > > > I agree with liyang to give a label to the RAW measure “Beta” or
>>> > others.
>>> > > >
>>> > > > I will improve it when I have time!
>>> > > >
>>> > > > 2016-12-19 10:21 GMT+08:00 Li Yang <li...@apache.org>:
>>> > > >
>>> > > > > Or display the Raw with "Beta" or "Experimental" to warn user
>>> that it
>>> > > is
>>> > > > > not a mature feature?
>>> > > > >
>>> > > > > On Fri, Dec 16, 2016 at 12:30 PM, 康凯森 <ka...@qq.com> wrote:
>>> > > > >
>>> > > > > > +1.
>>> > > > > > But the "RAW" measure is still some useful, we could improve it
>>> > next
>>> > > > year
>>> > > > > > when we have time.
>>> > > > > >
>>> > > > > >
>>> > > > > > ------------------ 原始邮件 ------------------
>>> > > > > > 发件人: "ShaoFeng Shi";<sh...@apache.org>;
>>> > > > > > 发送时间: 2016年12月15日(星期四) 中午12:05
>>> > > > > > 收件人: "dev"<de...@kylin.apache.org>;
>>> > > > > >
>>> > > > > > 主题: [Discuss] Disable/hide "RAW" measure in Kylin web GUI
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > > Hello developers and users,
>>> > > > > >
>>> > > > > > I have a proposal want to disucss here, which is about the
>>> "RAW"
>>> > > > measure
>>> > > > > in
>>> > > > > > Kylin.
>>> > > > > >
>>> > > > > > The RAW measure was developed to solve the requirement for
>>> getting
>>> > > raw
>>> > > > > data
>>> > > > > > when user drill down from high level to low levels. It's
>>> > performance
>>> > > > > would
>>> > > > > > be much better than fetching from source like Hive, so some
>>> users
>>> > > like
>>> > > > > it.
>>> > > > > > This blog introduces it:
>>> > > > > > https://kylin.apache.org/blog/2016/05/29/raw-measure-in-kylin/
>>> > > > > >
>>> > > > > > While it has some limitations:
>>> > > > > > 1) always use dictionary encoding, which means it couldn't
>>> support
>>> > > UHC;
>>> > > > > > since raw columns usually be transaction IDs, numbers, etc, the
>>> > > > building
>>> > > > > > cost is much higher than ordinary dimensions;
>>> > > > > > 2) the raw messages for a dimension combination are persisted
>>> in
>>> > one
>>> > > > big
>>> > > > > > cell,; When too many rows dumped in one cell, will get
>>> > BufferOverflow
>>> > > > > > error. This couldn't be predicted, so for an modeler or
>>> analyst he
>>> > > > > doesn't
>>> > > > > > know whether this feature could work when he creates the cube.
>>> > > > > > 3) seems no people maintains it;
>>> > > > > >
>>> > > > > > Based on above, I propose hiding this measure on Web GUI by
>>> > default,
>>> > > so
>>> > > > > > avoiding confusing user; If someone hear and want to use this,
>>> he
>>> > > still
>>> > > > > can
>>> > > > > > enable it by simply set in kylin.properties.
>>> > > > > >
>>> > > > > > Any input is welcomed.
>>> > > > > >
>>> > > > > > --
>>> > > > > > Best regards,
>>> > > > > >
>>> > > > > > Shaofeng Shi 史少锋
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>>
>>>
>>> --
>>> Best regards,
>>>
>>> Shaofeng Shi 史少锋
>>>
>>
>>
>
--
Best regards,
Shaofeng Shi 史少锋
Re: [Discuss] Disable/hide "RAW" measure in Kylin web GUI
Posted by Billy Liu <bi...@apache.org>.
+1 to turn this feature off by default. The advanced user could enable it
also.
2017-11-27 13:53 GMT+08:00 Luke Han <lu...@gmail.com>:
> +1 to remove it from new release, people could backport to new version
> with previous code
>
>
> Best Regards!
> ---------------------
>
> Luke Han
>
> On Sun, Nov 26, 2017 at 9:40 PM, ShaoFeng Shi <sh...@apache.org>
> wrote:
>
>> Last year I raised this discussion but didn't have follow-up action.
>>
>> Now we see there are still some new users misuse this feature, and then
>> face performance and maintenance issues.
>>
>> In Kylin 2.1, the new "query pushdown" feature can forward the Cube
>> unmatched queries to alternative query engines like Hive / SparkSQL. The
>> raw data query is just such a scenario.
>>
>> So I think it is time to disable the RAW measure on Kylin now. JIRA
>> created for it: https://issues.apache.org/jira/browse/KYLIN-3062
>>
>> Please comment if you see any issue.
>>
>> 2016-12-19 22:03 GMT+08:00 Billy Liu <bi...@apache.org>:
>>
>> > The experimental mode is system wide feature toggle. I think case by
>> case
>> > is more flexible. Most new features could have toggles, default are off.
>> >
>> > 2016-12-19 21:40 GMT+08:00 Luke Han <lu...@gmail.com>:
>> >
>> > > Beta or Experimental will also bring confusing for most of users.
>> > >
>> > > Maybe we could have something called "expert" or "experimental" model
>> in
>> > > system configuration.
>> > >
>> > > User will not see such content since they will be hidden by default
>> but
>> > > admin could set it to true if they confident to enable such features.
>> > >
>> > > How do you think?
>> > >
>> > >
>> > > Best Regards!
>> > > ---------------------
>> > >
>> > > Luke Han
>> > >
>> > > On Mon, Dec 19, 2016 at 11:24 AM, Xiaoyu Wang <wa...@gmail.com>
>> > wrote:
>> > >
>> > > > I’m sorry for too long not maintain it!
>> > > >
>> > > > I agree with liyang to give a label to the RAW measure “Beta” or
>> > others.
>> > > >
>> > > > I will improve it when I have time!
>> > > >
>> > > > 2016-12-19 10:21 GMT+08:00 Li Yang <li...@apache.org>:
>> > > >
>> > > > > Or display the Raw with "Beta" or "Experimental" to warn user
>> that it
>> > > is
>> > > > > not a mature feature?
>> > > > >
>> > > > > On Fri, Dec 16, 2016 at 12:30 PM, 康凯森 <ka...@qq.com> wrote:
>> > > > >
>> > > > > > +1.
>> > > > > > But the "RAW" measure is still some useful, we could improve it
>> > next
>> > > > year
>> > > > > > when we have time.
>> > > > > >
>> > > > > >
>> > > > > > ------------------ 原始邮件 ------------------
>> > > > > > 发件人: "ShaoFeng Shi";<sh...@apache.org>;
>> > > > > > 发送时间: 2016年12月15日(星期四) 中午12:05
>> > > > > > 收件人: "dev"<de...@kylin.apache.org>;
>> > > > > >
>> > > > > > 主题: [Discuss] Disable/hide "RAW" measure in Kylin web GUI
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > Hello developers and users,
>> > > > > >
>> > > > > > I have a proposal want to disucss here, which is about the "RAW"
>> > > > measure
>> > > > > in
>> > > > > > Kylin.
>> > > > > >
>> > > > > > The RAW measure was developed to solve the requirement for
>> getting
>> > > raw
>> > > > > data
>> > > > > > when user drill down from high level to low levels. It's
>> > performance
>> > > > > would
>> > > > > > be much better than fetching from source like Hive, so some
>> users
>> > > like
>> > > > > it.
>> > > > > > This blog introduces it:
>> > > > > > https://kylin.apache.org/blog/2016/05/29/raw-measure-in-kylin/
>> > > > > >
>> > > > > > While it has some limitations:
>> > > > > > 1) always use dictionary encoding, which means it couldn't
>> support
>> > > UHC;
>> > > > > > since raw columns usually be transaction IDs, numbers, etc, the
>> > > > building
>> > > > > > cost is much higher than ordinary dimensions;
>> > > > > > 2) the raw messages for a dimension combination are persisted in
>> > one
>> > > > big
>> > > > > > cell,; When too many rows dumped in one cell, will get
>> > BufferOverflow
>> > > > > > error. This couldn't be predicted, so for an modeler or analyst
>> he
>> > > > > doesn't
>> > > > > > know whether this feature could work when he creates the cube.
>> > > > > > 3) seems no people maintains it;
>> > > > > >
>> > > > > > Based on above, I propose hiding this measure on Web GUI by
>> > default,
>> > > so
>> > > > > > avoiding confusing user; If someone hear and want to use this,
>> he
>> > > still
>> > > > > can
>> > > > > > enable it by simply set in kylin.properties.
>> > > > > >
>> > > > > > Any input is welcomed.
>> > > > > >
>> > > > > > --
>> > > > > > Best regards,
>> > > > > >
>> > > > > > Shaofeng Shi 史少锋
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi 史少锋
>>
>
>
Re: [Discuss] Disable/hide "RAW" measure in Kylin web GUI
Posted by Billy Liu <bi...@apache.org>.
+1 to turn this feature off by default. The advanced user could enable it
also.
2017-11-27 13:53 GMT+08:00 Luke Han <lu...@gmail.com>:
> +1 to remove it from new release, people could backport to new version
> with previous code
>
>
> Best Regards!
> ---------------------
>
> Luke Han
>
> On Sun, Nov 26, 2017 at 9:40 PM, ShaoFeng Shi <sh...@apache.org>
> wrote:
>
>> Last year I raised this discussion but didn't have follow-up action.
>>
>> Now we see there are still some new users misuse this feature, and then
>> face performance and maintenance issues.
>>
>> In Kylin 2.1, the new "query pushdown" feature can forward the Cube
>> unmatched queries to alternative query engines like Hive / SparkSQL. The
>> raw data query is just such a scenario.
>>
>> So I think it is time to disable the RAW measure on Kylin now. JIRA
>> created for it: https://issues.apache.org/jira/browse/KYLIN-3062
>>
>> Please comment if you see any issue.
>>
>> 2016-12-19 22:03 GMT+08:00 Billy Liu <bi...@apache.org>:
>>
>> > The experimental mode is system wide feature toggle. I think case by
>> case
>> > is more flexible. Most new features could have toggles, default are off.
>> >
>> > 2016-12-19 21:40 GMT+08:00 Luke Han <lu...@gmail.com>:
>> >
>> > > Beta or Experimental will also bring confusing for most of users.
>> > >
>> > > Maybe we could have something called "expert" or "experimental" model
>> in
>> > > system configuration.
>> > >
>> > > User will not see such content since they will be hidden by default
>> but
>> > > admin could set it to true if they confident to enable such features.
>> > >
>> > > How do you think?
>> > >
>> > >
>> > > Best Regards!
>> > > ---------------------
>> > >
>> > > Luke Han
>> > >
>> > > On Mon, Dec 19, 2016 at 11:24 AM, Xiaoyu Wang <wa...@gmail.com>
>> > wrote:
>> > >
>> > > > I’m sorry for too long not maintain it!
>> > > >
>> > > > I agree with liyang to give a label to the RAW measure “Beta” or
>> > others.
>> > > >
>> > > > I will improve it when I have time!
>> > > >
>> > > > 2016-12-19 10:21 GMT+08:00 Li Yang <li...@apache.org>:
>> > > >
>> > > > > Or display the Raw with "Beta" or "Experimental" to warn user
>> that it
>> > > is
>> > > > > not a mature feature?
>> > > > >
>> > > > > On Fri, Dec 16, 2016 at 12:30 PM, 康凯森 <ka...@qq.com> wrote:
>> > > > >
>> > > > > > +1.
>> > > > > > But the "RAW" measure is still some useful, we could improve it
>> > next
>> > > > year
>> > > > > > when we have time.
>> > > > > >
>> > > > > >
>> > > > > > ------------------ 原始邮件 ------------------
>> > > > > > 发件人: "ShaoFeng Shi";<sh...@apache.org>;
>> > > > > > 发送时间: 2016年12月15日(星期四) 中午12:05
>> > > > > > 收件人: "dev"<de...@kylin.apache.org>;
>> > > > > >
>> > > > > > 主题: [Discuss] Disable/hide "RAW" measure in Kylin web GUI
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > Hello developers and users,
>> > > > > >
>> > > > > > I have a proposal want to disucss here, which is about the "RAW"
>> > > > measure
>> > > > > in
>> > > > > > Kylin.
>> > > > > >
>> > > > > > The RAW measure was developed to solve the requirement for
>> getting
>> > > raw
>> > > > > data
>> > > > > > when user drill down from high level to low levels. It's
>> > performance
>> > > > > would
>> > > > > > be much better than fetching from source like Hive, so some
>> users
>> > > like
>> > > > > it.
>> > > > > > This blog introduces it:
>> > > > > > https://kylin.apache.org/blog/2016/05/29/raw-measure-in-kylin/
>> > > > > >
>> > > > > > While it has some limitations:
>> > > > > > 1) always use dictionary encoding, which means it couldn't
>> support
>> > > UHC;
>> > > > > > since raw columns usually be transaction IDs, numbers, etc, the
>> > > > building
>> > > > > > cost is much higher than ordinary dimensions;
>> > > > > > 2) the raw messages for a dimension combination are persisted in
>> > one
>> > > > big
>> > > > > > cell,; When too many rows dumped in one cell, will get
>> > BufferOverflow
>> > > > > > error. This couldn't be predicted, so for an modeler or analyst
>> he
>> > > > > doesn't
>> > > > > > know whether this feature could work when he creates the cube.
>> > > > > > 3) seems no people maintains it;
>> > > > > >
>> > > > > > Based on above, I propose hiding this measure on Web GUI by
>> > default,
>> > > so
>> > > > > > avoiding confusing user; If someone hear and want to use this,
>> he
>> > > still
>> > > > > can
>> > > > > > enable it by simply set in kylin.properties.
>> > > > > >
>> > > > > > Any input is welcomed.
>> > > > > >
>> > > > > > --
>> > > > > > Best regards,
>> > > > > >
>> > > > > > Shaofeng Shi 史少锋
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi 史少锋
>>
>
>
Re: [Discuss] Disable/hide "RAW" measure in Kylin web GUI
Posted by Luke Han <lu...@gmail.com>.
+1 to remove it from new release, people could backport to new version with
previous code
Best Regards!
---------------------
Luke Han
On Sun, Nov 26, 2017 at 9:40 PM, ShaoFeng Shi <sh...@apache.org>
wrote:
> Last year I raised this discussion but didn't have follow-up action.
>
> Now we see there are still some new users misuse this feature, and then
> face performance and maintenance issues.
>
> In Kylin 2.1, the new "query pushdown" feature can forward the Cube
> unmatched queries to alternative query engines like Hive / SparkSQL. The
> raw data query is just such a scenario.
>
> So I think it is time to disable the RAW measure on Kylin now. JIRA
> created for it: https://issues.apache.org/jira/browse/KYLIN-3062
>
> Please comment if you see any issue.
>
> 2016-12-19 22:03 GMT+08:00 Billy Liu <bi...@apache.org>:
>
> > The experimental mode is system wide feature toggle. I think case by case
> > is more flexible. Most new features could have toggles, default are off.
> >
> > 2016-12-19 21:40 GMT+08:00 Luke Han <lu...@gmail.com>:
> >
> > > Beta or Experimental will also bring confusing for most of users.
> > >
> > > Maybe we could have something called "expert" or "experimental" model
> in
> > > system configuration.
> > >
> > > User will not see such content since they will be hidden by default but
> > > admin could set it to true if they confident to enable such features.
> > >
> > > How do you think?
> > >
> > >
> > > Best Regards!
> > > ---------------------
> > >
> > > Luke Han
> > >
> > > On Mon, Dec 19, 2016 at 11:24 AM, Xiaoyu Wang <wa...@gmail.com>
> > wrote:
> > >
> > > > I’m sorry for too long not maintain it!
> > > >
> > > > I agree with liyang to give a label to the RAW measure “Beta” or
> > others.
> > > >
> > > > I will improve it when I have time!
> > > >
> > > > 2016-12-19 10:21 GMT+08:00 Li Yang <li...@apache.org>:
> > > >
> > > > > Or display the Raw with "Beta" or "Experimental" to warn user that
> it
> > > is
> > > > > not a mature feature?
> > > > >
> > > > > On Fri, Dec 16, 2016 at 12:30 PM, 康凯森 <ka...@qq.com> wrote:
> > > > >
> > > > > > +1.
> > > > > > But the "RAW" measure is still some useful, we could improve it
> > next
> > > > year
> > > > > > when we have time.
> > > > > >
> > > > > >
> > > > > > ------------------ 原始邮件 ------------------
> > > > > > 发件人: "ShaoFeng Shi";<sh...@apache.org>;
> > > > > > 发送时间: 2016年12月15日(星期四) 中午12:05
> > > > > > 收件人: "dev"<de...@kylin.apache.org>;
> > > > > >
> > > > > > 主题: [Discuss] Disable/hide "RAW" measure in Kylin web GUI
> > > > > >
> > > > > >
> > > > > >
> > > > > > Hello developers and users,
> > > > > >
> > > > > > I have a proposal want to disucss here, which is about the "RAW"
> > > > measure
> > > > > in
> > > > > > Kylin.
> > > > > >
> > > > > > The RAW measure was developed to solve the requirement for
> getting
> > > raw
> > > > > data
> > > > > > when user drill down from high level to low levels. It's
> > performance
> > > > > would
> > > > > > be much better than fetching from source like Hive, so some users
> > > like
> > > > > it.
> > > > > > This blog introduces it:
> > > > > > https://kylin.apache.org/blog/2016/05/29/raw-measure-in-kylin/
> > > > > >
> > > > > > While it has some limitations:
> > > > > > 1) always use dictionary encoding, which means it couldn't
> support
> > > UHC;
> > > > > > since raw columns usually be transaction IDs, numbers, etc, the
> > > > building
> > > > > > cost is much higher than ordinary dimensions;
> > > > > > 2) the raw messages for a dimension combination are persisted in
> > one
> > > > big
> > > > > > cell,; When too many rows dumped in one cell, will get
> > BufferOverflow
> > > > > > error. This couldn't be predicted, so for an modeler or analyst
> he
> > > > > doesn't
> > > > > > know whether this feature could work when he creates the cube.
> > > > > > 3) seems no people maintains it;
> > > > > >
> > > > > > Based on above, I propose hiding this measure on Web GUI by
> > default,
> > > so
> > > > > > avoiding confusing user; If someone hear and want to use this, he
> > > still
> > > > > can
> > > > > > enable it by simply set in kylin.properties.
> > > > > >
> > > > > > Any input is welcomed.
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > >
> > > > > > Shaofeng Shi 史少锋
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
Re: [Discuss] Disable/hide "RAW" measure in Kylin web GUI
Posted by Luke Han <lu...@gmail.com>.
+1 to remove it from new release, people could backport to new version with
previous code
Best Regards!
---------------------
Luke Han
On Sun, Nov 26, 2017 at 9:40 PM, ShaoFeng Shi <sh...@apache.org>
wrote:
> Last year I raised this discussion but didn't have follow-up action.
>
> Now we see there are still some new users misuse this feature, and then
> face performance and maintenance issues.
>
> In Kylin 2.1, the new "query pushdown" feature can forward the Cube
> unmatched queries to alternative query engines like Hive / SparkSQL. The
> raw data query is just such a scenario.
>
> So I think it is time to disable the RAW measure on Kylin now. JIRA
> created for it: https://issues.apache.org/jira/browse/KYLIN-3062
>
> Please comment if you see any issue.
>
> 2016-12-19 22:03 GMT+08:00 Billy Liu <bi...@apache.org>:
>
> > The experimental mode is system wide feature toggle. I think case by case
> > is more flexible. Most new features could have toggles, default are off.
> >
> > 2016-12-19 21:40 GMT+08:00 Luke Han <lu...@gmail.com>:
> >
> > > Beta or Experimental will also bring confusing for most of users.
> > >
> > > Maybe we could have something called "expert" or "experimental" model
> in
> > > system configuration.
> > >
> > > User will not see such content since they will be hidden by default but
> > > admin could set it to true if they confident to enable such features.
> > >
> > > How do you think?
> > >
> > >
> > > Best Regards!
> > > ---------------------
> > >
> > > Luke Han
> > >
> > > On Mon, Dec 19, 2016 at 11:24 AM, Xiaoyu Wang <wa...@gmail.com>
> > wrote:
> > >
> > > > I’m sorry for too long not maintain it!
> > > >
> > > > I agree with liyang to give a label to the RAW measure “Beta” or
> > others.
> > > >
> > > > I will improve it when I have time!
> > > >
> > > > 2016-12-19 10:21 GMT+08:00 Li Yang <li...@apache.org>:
> > > >
> > > > > Or display the Raw with "Beta" or "Experimental" to warn user that
> it
> > > is
> > > > > not a mature feature?
> > > > >
> > > > > On Fri, Dec 16, 2016 at 12:30 PM, 康凯森 <ka...@qq.com> wrote:
> > > > >
> > > > > > +1.
> > > > > > But the "RAW" measure is still some useful, we could improve it
> > next
> > > > year
> > > > > > when we have time.
> > > > > >
> > > > > >
> > > > > > ------------------ 原始邮件 ------------------
> > > > > > 发件人: "ShaoFeng Shi";<sh...@apache.org>;
> > > > > > 发送时间: 2016年12月15日(星期四) 中午12:05
> > > > > > 收件人: "dev"<de...@kylin.apache.org>;
> > > > > >
> > > > > > 主题: [Discuss] Disable/hide "RAW" measure in Kylin web GUI
> > > > > >
> > > > > >
> > > > > >
> > > > > > Hello developers and users,
> > > > > >
> > > > > > I have a proposal want to disucss here, which is about the "RAW"
> > > > measure
> > > > > in
> > > > > > Kylin.
> > > > > >
> > > > > > The RAW measure was developed to solve the requirement for
> getting
> > > raw
> > > > > data
> > > > > > when user drill down from high level to low levels. It's
> > performance
> > > > > would
> > > > > > be much better than fetching from source like Hive, so some users
> > > like
> > > > > it.
> > > > > > This blog introduces it:
> > > > > > https://kylin.apache.org/blog/2016/05/29/raw-measure-in-kylin/
> > > > > >
> > > > > > While it has some limitations:
> > > > > > 1) always use dictionary encoding, which means it couldn't
> support
> > > UHC;
> > > > > > since raw columns usually be transaction IDs, numbers, etc, the
> > > > building
> > > > > > cost is much higher than ordinary dimensions;
> > > > > > 2) the raw messages for a dimension combination are persisted in
> > one
> > > > big
> > > > > > cell,; When too many rows dumped in one cell, will get
> > BufferOverflow
> > > > > > error. This couldn't be predicted, so for an modeler or analyst
> he
> > > > > doesn't
> > > > > > know whether this feature could work when he creates the cube.
> > > > > > 3) seems no people maintains it;
> > > > > >
> > > > > > Based on above, I propose hiding this measure on Web GUI by
> > default,
> > > so
> > > > > > avoiding confusing user; If someone hear and want to use this, he
> > > still
> > > > > can
> > > > > > enable it by simply set in kylin.properties.
> > > > > >
> > > > > > Any input is welcomed.
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > >
> > > > > > Shaofeng Shi 史少锋
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>