You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by ro...@stratebi.com on 2018/04/02 16:00:20 UTC

Dimension update issue

Hi,

 

I have some questions about how Kylin refresh dimensions tables.  In case
you delete some instances of the dimension on hive table that are not used
by any fact in fact table, These values will be deleted from dimension table
on next cube segment built? We have to refresh old segments?

 

In our case we have added by mistake a lot of empty future days to a Date
dimension (80 years). Then we have built some days of the cube. However, as
this big date dimensions leads to performance issues, we have deleted most
of future years from Date dimension. However, after built a new segment of
the cube, the deleted rows on Hive table they have not been deleted in Kylin
cube table. How can we refresh the dimension table on Kylin properly?

 

Thanks in advance!

 

Roberto Tardío Olmos

Senior Big Data & Business Intelligence Consultant

Avenida de Brasil, 17, Planta 16.28020 Madrid

Fijo: 91.788.34.10




 

http://bigdata.stratebi.com/ 

 

http://www.stratebi.com

RE: Dimension update issue

Posted by ro...@stratebi.com.
Many thanks another time ShaoFeng. We have refreshed segments that were built with modified dimension table and now we see the changes on Date dimension table.

 

Kind regards

 

From: ShaoFeng Shi [mailto:shaofengshi@apache.org] 
Sent: martes, 3 de abril de 2018 4:22
To: user <us...@kylin.apache.org>
Subject: Re: Dimension update issue

 

Hi Roberto,

 

Kylin will take the snapshot for dimension tables, and persist the snapshots together with the cube segments.

 

If you have modified the dimension table in Hive, and expects to see the changes for history segments, the proper way is to refresh the history segment(s), which will refresh the snapshot.

 

Hope this clarifies your questions.

 

 

 

2018-04-03 0:00 GMT+08:00 <roberto.tardio@stratebi.com <ma...@stratebi.com> >:

Hi,

 

I have some questions about how Kylin refresh dimensions tables.  In case you delete some instances of the dimension on hive table that are not used by any fact in fact table, These values will be deleted from dimension table on next cube segment built? We have to refresh old segments?

 

In our case we have added by mistake a lot of empty future days to a Date dimension (80 years). Then we have built some days of the cube. However, as this big date dimensions leads to performance issues, we have deleted most of future years from Date dimension. However, after built a new segment of the cube, the deleted rows on Hive table they have not been deleted in Kylin cube table. How can we refresh the dimension table on Kylin properly?

 

Thanks in advance!

 

Roberto Tardío Olmos

Senior Big Data & Business Intelligence Consultant

Avenida de Brasil, 17 <https://maps.google.com/?q=Avenida+de+Brasil,+17&entry=gmail&source=g> , Planta 16.28020 Madrid

Fijo: 91.788.34.10




 

http://bigdata.stratebi.com/ 

 

http://www.stratebi.com 

 





 

-- 

Best regards,

 

Shaofeng Shi 史少锋

Re: Dimension update issue

Posted by ShaoFeng Shi <sh...@apache.org>.
Hi Roberto,

Kylin will take the snapshot for dimension tables, and persist the
snapshots together with the cube segments.

If you have modified the dimension table in Hive, and expects to see the
changes for history segments, the proper way is to refresh the history
segment(s), which will refresh the snapshot.

Hope this clarifies your questions.



2018-04-03 0:00 GMT+08:00 <ro...@stratebi.com>:

> Hi,
>
>
>
> I have some questions about how Kylin refresh dimensions tables.  In case
> you delete some instances of the dimension on hive table that are not used
> by any fact in fact table, These values will be deleted from dimension
> table on next cube segment built? We have to refresh old segments?
>
>
>
> In our case we have added by mistake a lot of empty future days to a Date
> dimension (80 years). Then we have built some days of the cube. However, as
> this big date dimensions leads to performance issues, we have deleted most
> of future years from Date dimension. However, after built a new segment of
> the cube, the deleted rows on Hive table they have not been deleted in
> Kylin cube table. How can we refresh the dimension table on Kylin properly?
>
>
>
> Thanks in advance!
>
>
>
> *Roberto Tardío Olmos*
>
> *Senior Big Data & Business Intelligence Consultant*
>
> Avenida de Brasil, 17
> <https://maps.google.com/?q=Avenida+de+Brasil,+17&entry=gmail&source=g>,
> Planta 16.28020 Madrid
>
> Fijo: 91.788.34.10
>
>
> [image:
> http://www.stratebi.com/image/layout_set_logo?img_id=21615&t=1486381163544]
>
>
>
> http://bigdata.stratebi.com/
>
>
>
> http://www.stratebi.com
>
>
>



-- 
Best regards,

Shaofeng Shi 史少锋