You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by Warren Tracey <WT...@mrpg.com> on 2016/09/07 06:01:18 UTC
Kylin: On the Fly Calcs
Hi
I tried to get hold of the Kylin Community, but cannot seem to locate a forum for Kylin.
I am emailing from Durban, South Africa. The problem I have is that I have 4 BI Tools sitting on top of Hadoop being used across the company, namely OBIEE, Tableau, Qlikview and Power BI. The problem with so many BI Tools is multiple semantic layers which is a gross duplication of effort and a single version of the truth is not maintained.
I would ultimately like 1 single, unified semantic layer from which each of these tools would access. I was hoping Kylin was that tool, but I understand that on the fly calculations like ratios are not included in Kylin e.g. Gross Profit %. The article were I saw this restriction is explained below.
https://community.hortonworks.com/articles/8305/a-quick-skinny-on-apache-kylin.html
Would you kindly advise how Kylin deals with these on the fly calcs, if it doesn't have the feature then does it use another Open Source Technology?
Regards
Warren Tracey
T:
E:
W: http://www.mrpricegroup.com
The information contained in this communication from @|from|email|@ is confidential and may be legally privileged. This email is intended solely for use by the addressee and others authorized to receive it, and should not be read if delivered in error. If you are not the addressee you are hereby notified that any disclosure, copying, distribution or action taken or omitted in reliance of the contents of this information is strictly prohibited and may be unlawful. Please notify the sender immediately if the e-mail has unintentionally reached you. The email legal notice that governs the transmission of this electronic communication is available online at Mr Price Group Limited (http://www.mrpricegroup.com/LegalNotice.aspx). If you cannot access the legal notice, kindly contact us on +27 800 21 2535 and a copy will be provided to you.
Mr Price Group is an Authorised Financial Services and Credit Provider: FSP31450 and NCRCP46.
Please consider the environment before printing this email.
Re: Kylin: On the Fly Calcs
Posted by ShaoFeng Shi <sh...@apache.org>.
Hi Warren,
We don't have a forum so far; I read the article you mentioned very early,
and I put 6 or 7 comments there, please expand the "comments" section.
For the ratio calculation, could you please give a specific SQL as a
sample? Thanks.
2016-09-07 14:01 GMT+08:00 Warren Tracey <WT...@mrpg.com>:
> Hi
>
>
>
> I tried to get hold of the Kylin Community, but cannot seem to locate a
> forum for Kylin.
>
>
>
> I am emailing from Durban, South Africa. The problem I have is that I
> have 4 BI Tools sitting on top of Hadoop being used across the company,
> namely OBIEE, Tableau, Qlikview and Power BI. The problem with so many BI
> Tools is multiple semantic layers which is a gross duplication of effort
> and a single version of the truth is not maintained.
>
>
>
> I would ultimately like 1 single, unified semantic layer from which each
> of these tools would access. I was hoping Kylin was that tool, but I
> understand that on the fly calculations like ratios are not included in
> Kylin e.g. Gross Profit %. The article were I saw this restriction is
> explained below.
>
>
>
> https://community.hortonworks.com/articles/8305/a-quick-
> skinny-on-apache-kylin.html
>
>
>
> Would you kindly advise how Kylin deals with these on the fly calcs, if it
> doesn’t have the feature then does it use another Open Source Technology?
>
>
>
> Regards
>
> Warren Tracey
>
>
> [image: Mr PriceGroup Limited]
> Warren Tracey
> Portfolio Manager
> mrp IT
> T:
> E: wtracey@mrpg.com
> W: http://www.mrpricegroup.com
>
>
> The information contained in this communication from wtracey@mrpg.com is
> confidential and may be legally privileged. This email is intended solely
> for use by the addressee and others authorized to receive it, and should
> not be read if delivered in error. If you are not the addressee you are
> hereby notified that any disclosure, copying, distribution or action taken
> or omitted in reliance of the contents of this information is strictly
> prohibited and may be unlawful. Please notify the sender immediately if the
> e-mail has unintentionally reached you. The email legal notice that governs
> the transmission of this electronic communication is available online at Mr
> Price Group Limited (http://www.mrpricegroup.com/LegalNotice.aspx). If
> you cannot access the legal notice, kindly contact us on +27 800 21 2535
> and a copy will be provided to you.
>
> Mr Price Group is an Authorised Financial Services and Credit Provider:
> FSP31450 and NCRCP46.
>
> PPlease consider the environment before printing this email.
>
--
Best regards,
Shaofeng Shi
Re: Kylin: On the Fly Calcs
Posted by ShaoFeng Shi <sh...@apache.org>.
Hi Warren,
Kylin is building cube on Hadoop; it support common aggregations like SUM,
MIN, MAX, etc; The aggregating column need be washed and pre-processed
during the ETL process, so that Kylin can focus on using Hadoop to do
massive aggregation;
Usually for the "on the fly" calculation, you need do that before enter
into cube/kylin, you can easily do this with a hive view, for example:
create view my_view as
select col_a, col_b,
case when fact1.column5 <> 0 then ( (fact1.column1 + fact2.column2 –
fact3.column3) / (fact1.column5)) * 100 else 0 end" as xyz
from fact_table;
Then in Kylin's cube, use the new column "xyz" in aggregation like SUM(),
MIN(), etc.
Hope it answers your question;
2016-09-07 14:01 GMT+08:00 Warren Tracey <WT...@mrpg.com>:
> Hi
>
>
>
> I tried to get hold of the Kylin Community, but cannot seem to locate a
> forum for Kylin.
>
>
>
> I am emailing from Durban, South Africa. The problem I have is that I
> have 4 BI Tools sitting on top of Hadoop being used across the company,
> namely OBIEE, Tableau, Qlikview and Power BI. The problem with so many BI
> Tools is multiple semantic layers which is a gross duplication of effort
> and a single version of the truth is not maintained.
>
>
>
> I would ultimately like 1 single, unified semantic layer from which each
> of these tools would access. I was hoping Kylin was that tool, but I
> understand that on the fly calculations like ratios are not included in
> Kylin e.g. Gross Profit %. The article were I saw this restriction is
> explained below.
>
>
>
> https://community.hortonworks.com/articles/8305/a-quick-
> skinny-on-apache-kylin.html
>
>
>
> Would you kindly advise how Kylin deals with these on the fly calcs, if it
> doesn’t have the feature then does it use another Open Source Technology?
>
>
>
> Regards
>
> Warren Tracey
>
>
> [image: Mr PriceGroup Limited]
> Warren Tracey
> Portfolio Manager
> mrp IT
> T:
> E: wtracey@mrpg.com
> W: http://www.mrpricegroup.com
>
>
> The information contained in this communication from wtracey@mrpg.com is
> confidential and may be legally privileged. This email is intended solely
> for use by the addressee and others authorized to receive it, and should
> not be read if delivered in error. If you are not the addressee you are
> hereby notified that any disclosure, copying, distribution or action taken
> or omitted in reliance of the contents of this information is strictly
> prohibited and may be unlawful. Please notify the sender immediately if the
> e-mail has unintentionally reached you. The email legal notice that governs
> the transmission of this electronic communication is available online at Mr
> Price Group Limited (http://www.mrpricegroup.com/LegalNotice.aspx). If
> you cannot access the legal notice, kindly contact us on +27 800 21 2535
> and a copy will be provided to you.
>
> Mr Price Group is an Authorised Financial Services and Credit Provider:
> FSP31450 and NCRCP46.
>
> PPlease consider the environment before printing this email.
>
--
Best regards,
Shaofeng Shi