You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by 钱巧能 <co...@126.com> on 2016/02/05 06:19:23 UTC
Kylin cube segment deletion & auto data retention
Dear Kylin Community,
Its Kalin, I am quite excited to join and willing to contribute to Kylin project. I work at eBay, as you all know that eBay is a big contributor and also customer of Kylin. Recently we have some requirement to retire some old data for customer. And now we need to do it with some manual steps. It is better to provide this as a service with GUI.
And Kylin's strategy is to use space to trade of query time, thus Kylin cubes consume lots of HBase space. And there are always the case that customer doesn't need to query old data. So enable Kylin to do auto data retention is a valuable feature. With this feature, user can define its data retention rule when create the project. Then Kylin server will automatically purge relative segments at certain intervals.
I opened 2 JIRA tickets for these 2 issues, would like to see your comments. Thanks!
KYLIN-1406Kylin Auto Data Retention
KYLIN-1404Provide Services to purge Segments for Kylin Admin
Re: Kylin cube segment deletion & auto data retention
Posted by hongbin ma <ma...@apache.org>.
and for KYLIN-1404 you don't need another service, just extend current
"purge" function for disabled cubes.
On Fri, Feb 5, 2016 at 1:47 PM, hongbin ma <ma...@apache.org> wrote:
> +1 on KYLIN-1404 Provide Services to purge Segments for Kylin Admin,
> however KYLIN-1406 seems already implemented.
>
> On Fri, Feb 5, 2016 at 1:28 PM, Hao Chen <cn...@gmail.com> wrote:
>
>> +1 Kylin should support Cube TTL
>>
>> Thanks,
>> Hao
>>
>> > On Feb 5, 2016, at 1:19 PM, 钱巧能 <co...@126.com> wrote:
>> >
>> > Dear Kylin Community,
>> > Its Kalin, I am quite excited to join and willing to contribute to
>> Kylin project. I work at eBay, as you all know that eBay is a big
>> contributor and also customer of Kylin. Recently we have some requirement
>> to retire some old data for customer. And now we need to do it with some
>> manual steps. It is better to provide this as a service with GUI.
>> > And Kylin's strategy is to use space to trade of query time, thus Kylin
>> cubes consume lots of HBase space. And there are always the case that
>> customer doesn't need to query old data. So enable Kylin to do auto data
>> retention is a valuable feature. With this feature, user can define its
>> data retention rule when create the project. Then Kylin server will
>> automatically purge relative segments at certain intervals.
>> > I opened 2 JIRA tickets for these 2 issues, would like to see your
>> comments. Thanks!
>> > KYLIN-1406Kylin Auto Data Retention
>> > KYLIN-1404Provide Services to purge Segments for Kylin Admin
>>
>
>
>
> --
> Regards,
>
> *Bin Mahone | 马洪宾*
> Apache Kylin: http://kylin.io
> Github: https://github.com/binmahone
>
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone
Re: Kylin cube segment deletion & auto data retention
Posted by hongbin ma <ma...@apache.org>.
+1 on KYLIN-1404 Provide Services to purge Segments for Kylin Admin,
however KYLIN-1406 seems already implemented.
On Fri, Feb 5, 2016 at 1:28 PM, Hao Chen <cn...@gmail.com> wrote:
> +1 Kylin should support Cube TTL
>
> Thanks,
> Hao
>
> > On Feb 5, 2016, at 1:19 PM, 钱巧能 <co...@126.com> wrote:
> >
> > Dear Kylin Community,
> > Its Kalin, I am quite excited to join and willing to contribute to Kylin
> project. I work at eBay, as you all know that eBay is a big contributor and
> also customer of Kylin. Recently we have some requirement to retire some
> old data for customer. And now we need to do it with some manual steps. It
> is better to provide this as a service with GUI.
> > And Kylin's strategy is to use space to trade of query time, thus Kylin
> cubes consume lots of HBase space. And there are always the case that
> customer doesn't need to query old data. So enable Kylin to do auto data
> retention is a valuable feature. With this feature, user can define its
> data retention rule when create the project. Then Kylin server will
> automatically purge relative segments at certain intervals.
> > I opened 2 JIRA tickets for these 2 issues, would like to see your
> comments. Thanks!
> > KYLIN-1406Kylin Auto Data Retention
> > KYLIN-1404Provide Services to purge Segments for Kylin Admin
>
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone
Re: Kylin cube segment deletion & auto data retention
Posted by Hao Chen <cn...@gmail.com>.
+1 Kylin should support Cube TTL
Thanks,
Hao
> On Feb 5, 2016, at 1:19 PM, 钱巧能 <co...@126.com> wrote:
>
> Dear Kylin Community,
> Its Kalin, I am quite excited to join and willing to contribute to Kylin project. I work at eBay, as you all know that eBay is a big contributor and also customer of Kylin. Recently we have some requirement to retire some old data for customer. And now we need to do it with some manual steps. It is better to provide this as a service with GUI.
> And Kylin's strategy is to use space to trade of query time, thus Kylin cubes consume lots of HBase space. And there are always the case that customer doesn't need to query old data. So enable Kylin to do auto data retention is a valuable feature. With this feature, user can define its data retention rule when create the project. Then Kylin server will automatically purge relative segments at certain intervals.
> I opened 2 JIRA tickets for these 2 issues, would like to see your comments. Thanks!
> KYLIN-1406Kylin Auto Data Retention
> KYLIN-1404Provide Services to purge Segments for Kylin Admin