You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by ShaoFeng Shi <sh...@apache.org> on 2019/07/31 05:16:03 UTC
Re: does anyone tell me why raw measure need dictionary?
This is a limitation of the raw measure; It uses a dictionary to compress
the values, but that may make the dictionary very big. This is one of the
reasons that Kylin mark the raw measure as deprecated.
Best regards,
Shaofeng Shi 史少锋
Apache Kylin PMC
Email: shaofengshi@apache.org
Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscribe@kylin.apache.org
Join Kylin dev mail group: dev-subscribe@kylin.apache.org
you Zhuang <zh...@gmail.com> 于2019年7月15日周一 下午11:27写道：
> I use raw measure to serve original records consults. But why does raw
> measure need dictionary? If I have a lot of raw measures, performance is
> slow. I see code below :
>
> @Override
> public List<TblColRef> getColumnsNeedDictionary(FunctionDesc functionDesc) {
> TblColRef literalCol = functionDesc.getParameter().getColRefs().get(0);
> return Collections.singletonList(literalCol);
> }
>
>
> My raw measures are always double, not big. So I don’t need dictionary at
> all.
>
> I use raw measures for historical reasons, so I expect someone to tell me
> why and how I can modify to use raw bytes.
>
Re: does anyone tell me why raw measure need dictionary?
Posted by ShaoFeng Shi <sh...@apache.org>.
Awesome！
Best regards,
Shaofeng Shi 史少锋
Apache Kylin PMC
Email: shaofengshi@apache.org
Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscribe@kylin.apache.org
Join Kylin dev mail group: dev-subscribe@kylin.apache.org
you Zhuang <zh...@gmail.com> 于2019年7月31日周三 下午9:19写道：
> Thanks ShaoFeng, I have copy a rawv2 measure without dictionary instead,
> all memory metrics have been around normal level since then.
>
> On Jul 31, 2019, at 1:16 PM, ShaoFeng Shi <sh...@apache.org> wrote:
>
> This is a limitation of the raw measure; It uses a dictionary to compress
> the values, but that may make the dictionary very big. This is one of the
> reasons that Kylin mark the raw measure as deprecated.
>
> Best regards,
>
> Shaofeng Shi 史少锋
> Apache Kylin PMC
> Email: shaofengshi@apache.org
>
> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
> Join Kylin user mail group: user-subscribe@kylin.apache.org
> Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>
>
>
>
> you Zhuang <zh...@gmail.com> 于2019年7月15日周一 下午11:27写道：
>
>> I use raw measure to serve original records consults. But why does raw
>> measure need dictionary? If I have a lot of raw measures, performance is
>> slow. I see code below :
>>
>> @Override
>> public List<TblColRef> getColumnsNeedDictionary(FunctionDesc functionDesc) {
>> TblColRef literalCol = functionDesc.getParameter().getColRefs().get(0);
>> return Collections.singletonList(literalCol);
>> }
>>
>>
>> My raw measures are always double, not big. So I don’t need dictionary at
>> all.
>>
>> I use raw measures for historical reasons, so I expect someone to tell me
>> why and how I can modify to use raw bytes.
>>
>
>
Re: does anyone tell me why raw measure need dictionary?
Posted by you Zhuang <zh...@gmail.com>.
Thanks ShaoFeng, I have copy a rawv2 measure without dictionary instead, all memory metrics have been around normal level since then.
> On Jul 31, 2019, at 1:16 PM, ShaoFeng Shi <sh...@apache.org> wrote:
>
> This is a limitation of the raw measure; It uses a dictionary to compress the values, but that may make the dictionary very big. This is one of the reasons that Kylin mark the raw measure as deprecated.
>
> Best regards,
>
> Shaofeng Shi 史少锋
> Apache Kylin PMC
> Email: shaofengshi@apache.org <ma...@apache.org>
>
> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html <https://kylin.apache.org/docs/gettingstarted/faq.html>
> Join Kylin user mail group: user-subscribe@kylin.apache.org <ma...@kylin.apache.org>
> Join Kylin dev mail group: dev-subscribe@kylin.apache.org <ma...@kylin.apache.org>
>
>
>
>
> you Zhuang <zhuangzixiaoyou@gmail.com <ma...@gmail.com>> 于2019年7月15日周一 下午11:27写道：
> I use raw measure to serve original records consults. But why does raw measure need dictionary? If I have a lot of raw measures, performance is slow. I see code below :
>
> @Override
> public List<TblColRef> getColumnsNeedDictionary(FunctionDesc functionDesc) {
> TblColRef literalCol = functionDesc.getParameter().getColRefs().get(0);
> return Collections.singletonList(literalCol);
> }
>
> My raw measures are always double, not big. So I don’t need dictionary at all.
>
> I use raw measures for historical reasons, so I expect someone to tell me why and how I can modify to use raw bytes.