You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by Shrikant Bang <b....@gmail.com> on 2018/08/20 09:59:43 UTC
Changes In Dimensions And Historical Cube's Data
Hi Team,
We have a use case where dimension's data may get modified ( slowly
changing dimension (SCD)).
Here is example:
Fact :
user_activity (
user_id STRING,
country_code INT,
....
)
Dimension:
country_dim (
country_code INT,
country_name STRING ,
....
)
Let's say we have mapped country code 1 to 'example_country1' and keep
building cube for years. Now country code 1 is assigned to
'new_country_code1'.
I have below query:
1. Is there any way to update the cube or it has to rebuild for all
past time segment?
2. Can we join cube data with other dimensions on runtime ( changing
dimensions table) , something like look ups?
Thank You,
Shrikant Bang.
Re: Changes In Dimensions And Historical Cube's Data
Posted by Billy Liu <bi...@apache.org>.
Hello Shrikant,
I think there are some new updates on lookup table capability. Check
out https://issues.apache.org/jira/browse/KYLIN-3221
Each dimension table is stored as snapshot(if you are using derived
dimension, not normal dimension), and be connected with the correspond
segment. With KYLIN-3221, you could refresh the existing lookup table
as needed.
With Warm regards
Billy Liu
Shrikant Bang <b....@gmail.com> 于2018年8月21日周二 上午11:26写道：
>
> Hello Kylin Users,
>
> I found a similar mail thread of Dec. 2015 ( http://apache-kylin.74782.x6.nabble.com/Incremental-builds-assumptions-and-clarifications-td2736.html ).
> Can someone please confirm if this understanding is applicable for current version (v2.4) of Apache Kylin also?
>
> Thank You,
> Shrikant Bang.
>
> On Mon, Aug 20, 2018 at 3:29 PM Shrikant Bang <b....@gmail.com> wrote:
>>
>> Hi Team,
>>
>> We have a use case where dimension's data may get modified ( slowly changing dimension (SCD)).
>>
>> Here is example:
>>
>> Fact :
>> user_activity (
>> user_id STRING,
>> country_code INT,
>> ....
>> )
>>
>> Dimension:
>> country_dim (
>> country_code INT,
>> country_name STRING ,
>> ....
>> )
>>
>>
>> Let's say we have mapped country code 1 to 'example_country1' and keep building cube for years. Now country code 1 is assigned to 'new_country_code1'.
>>
>>
>> I have below query:
>>
>> Is there any way to update the cube or it has to rebuild for all past time segment?
>> Can we join cube data with other dimensions on runtime ( changing dimensions table) , something like look ups?
>>
>>
>> Thank You,
>> Shrikant Bang.
>>
>>
>>
>>
Re: Changes In Dimensions And Historical Cube's Data
Posted by Billy Liu <bi...@apache.org>.
Hello Shrikant,
I think there are some new updates on lookup table capability. Check
out https://issues.apache.org/jira/browse/KYLIN-3221
Each dimension table is stored as snapshot(if you are using derived
dimension, not normal dimension), and be connected with the correspond
segment. With KYLIN-3221, you could refresh the existing lookup table
as needed.
With Warm regards
Billy Liu
Shrikant Bang <b....@gmail.com> 于2018年8月21日周二 上午11:26写道：
>
> Hello Kylin Users,
>
> I found a similar mail thread of Dec. 2015 ( http://apache-kylin.74782.x6.nabble.com/Incremental-builds-assumptions-and-clarifications-td2736.html ).
> Can someone please confirm if this understanding is applicable for current version (v2.4) of Apache Kylin also?
>
> Thank You,
> Shrikant Bang.
>
> On Mon, Aug 20, 2018 at 3:29 PM Shrikant Bang <b....@gmail.com> wrote:
>>
>> Hi Team,
>>
>> We have a use case where dimension's data may get modified ( slowly changing dimension (SCD)).
>>
>> Here is example:
>>
>> Fact :
>> user_activity (
>> user_id STRING,
>> country_code INT,
>> ....
>> )
>>
>> Dimension:
>> country_dim (
>> country_code INT,
>> country_name STRING ,
>> ....
>> )
>>
>>
>> Let's say we have mapped country code 1 to 'example_country1' and keep building cube for years. Now country code 1 is assigned to 'new_country_code1'.
>>
>>
>> I have below query:
>>
>> Is there any way to update the cube or it has to rebuild for all past time segment?
>> Can we join cube data with other dimensions on runtime ( changing dimensions table) , something like look ups?
>>
>>
>> Thank You,
>> Shrikant Bang.
>>
>>
>>
>>
Re: Changes In Dimensions And Historical Cube's Data
Posted by Shrikant Bang <b....@gmail.com>.
Hello Kylin Users,
I found a similar mail thread of Dec. 2015 (
http://apache-kylin.74782.x6.nabble.com/Incremental-builds-assumptions-and-clarifications-td2736.html
).
Can someone please confirm if this understanding is applicable for
current version (v2.4) of Apache Kylin also?
Thank You,
Shrikant Bang.
On Mon, Aug 20, 2018 at 3:29 PM Shrikant Bang <b....@gmail.com>
wrote:
> Hi Team,
>
> We have a use case where dimension's data may get modified ( slowly
> changing dimension (SCD)).
>
> Here is example:
>
> Fact :
> user_activity (
> user_id STRING,
> country_code INT,
> ....
> )
>
> Dimension:
> country_dim (
> country_code INT,
> country_name STRING ,
> ....
> )
>
>
> Let's say we have mapped country code 1 to 'example_country1' and keep
> building cube for years. Now country code 1 is assigned to
> 'new_country_code1'.
>
>
> I have below query:
>
> 1. Is there any way to update the cube or it has to rebuild for all
> past time segment?
> 2. Can we join cube data with other dimensions on runtime ( changing
> dimensions table) , something like look ups?
>
>
> Thank You,
> Shrikant Bang.
>
>
>
>
>
Re: Changes In Dimensions And Historical Cube's Data
Posted by Shrikant Bang <b....@gmail.com>.
Hello Kylin Users,
I found a similar mail thread of Dec. 2015 (
http://apache-kylin.74782.x6.nabble.com/Incremental-builds-assumptions-and-clarifications-td2736.html
).
Can someone please confirm if this understanding is applicable for
current version (v2.4) of Apache Kylin also?
Thank You,
Shrikant Bang.
On Mon, Aug 20, 2018 at 3:29 PM Shrikant Bang <b....@gmail.com>
wrote:
> Hi Team,
>
> We have a use case where dimension's data may get modified ( slowly
> changing dimension (SCD)).
>
> Here is example:
>
> Fact :
> user_activity (
> user_id STRING,
> country_code INT,
> ....
> )
>
> Dimension:
> country_dim (
> country_code INT,
> country_name STRING ,
> ....
> )
>
>
> Let's say we have mapped country code 1 to 'example_country1' and keep
> building cube for years. Now country code 1 is assigned to
> 'new_country_code1'.
>
>
> I have below query:
>
> 1. Is there any way to update the cube or it has to rebuild for all
> past time segment?
> 2. Can we join cube data with other dimensions on runtime ( changing
> dimensions table) , something like look ups?
>
>
> Thank You,
> Shrikant Bang.
>
>
>
>
>