You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by "Shi, Shaofeng" <sh...@ebay.com> on 2015/04/02 08:26:44 UTC
Command to drop all Kylin intermediate flat tables
Kylin creates an intermediate flat table when building a cube each time; Due to some reason the table wasn’t dropped automatically after cube be built; So after some period of time, we suggest you to drop these tables to cleanup the HDFS space; Here is a simple command to drop all these temp tables in batch (make sure there is no cube being built):
hive -e 'show tables' | grep kylin_intermediate_ | xargs -I '{}' hive -e 'drop table {}'
Re: Command to drop all Kylin intermediate flat tables
Posted by Luke Han <lu...@gmail.com>.
JIRA already there but putted to future release:
https://issues.apache.org/jira/browse/KYLIN-589
Welcome if someone from community could contribution on this.
Thanks.
Best Regards!
---------------------
Luke Han
2015-04-07 10:48 GMT+08:00 Li Yang <li...@apache.org>:
> We will drop the intermediate hive table in the end (when reach 1.0?).
> Right now, it is kept for troubleshooting purpose.
>
> On Thu, Apr 2, 2015 at 3:03 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
>
> > Merge cube segments will base on the generated cuboid files, not the
> > intermediate hive table, so it is safe to drop the hive table once the
> > base cuboid be calculated;
> >
> > On 4/2/15, 2:58 PM, "dong wang" <el...@gmail.com> wrote:
> >
> > >for this, I have a question that after building the cube segment
> > >successfully, whether the intermediate tables are useful or not when
> > >MERGING some segments?
> > >
> > >2015-04-02 14:26 GMT+08:00 Shi, Shaofeng <sh...@ebay.com>:
> > >
> > >> Kylin creates an intermediate flat table when building a cube each
> time;
> > >> Due to some reason the table wasn¹t dropped automatically after cube
> be
> > >> built; So after some period of time, we suggest you to drop these
> > >>tables to
> > >> cleanup the HDFS space; Here is a simple command to drop all these
> temp
> > >> tables in batch (make sure there is no cube being built):
> > >>
> > >> hive -e 'show tables' | grep kylin_intermediate_ | xargs -I '{}' hive
> -e
> > >> 'drop table {}'
> > >>
> > >>
> > >>
> >
> >
>
Re: Command to drop all Kylin intermediate flat tables
Posted by Li Yang <li...@apache.org>.
We will drop the intermediate hive table in the end (when reach 1.0?).
Right now, it is kept for troubleshooting purpose.
On Thu, Apr 2, 2015 at 3:03 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
> Merge cube segments will base on the generated cuboid files, not the
> intermediate hive table, so it is safe to drop the hive table once the
> base cuboid be calculated;
>
> On 4/2/15, 2:58 PM, "dong wang" <el...@gmail.com> wrote:
>
> >for this, I have a question that after building the cube segment
> >successfully, whether the intermediate tables are useful or not when
> >MERGING some segments?
> >
> >2015-04-02 14:26 GMT+08:00 Shi, Shaofeng <sh...@ebay.com>:
> >
> >> Kylin creates an intermediate flat table when building a cube each time;
> >> Due to some reason the table wasn¹t dropped automatically after cube be
> >> built; So after some period of time, we suggest you to drop these
> >>tables to
> >> cleanup the HDFS space; Here is a simple command to drop all these temp
> >> tables in batch (make sure there is no cube being built):
> >>
> >> hive -e 'show tables' | grep kylin_intermediate_ | xargs -I '{}' hive -e
> >> 'drop table {}'
> >>
> >>
> >>
>
>
Re: Command to drop all Kylin intermediate flat tables
Posted by "Shi, Shaofeng" <sh...@ebay.com>.
Merge cube segments will base on the generated cuboid files, not the
intermediate hive table, so it is safe to drop the hive table once the
base cuboid be calculated;
On 4/2/15, 2:58 PM, "dong wang" <el...@gmail.com> wrote:
>for this, I have a question that after building the cube segment
>successfully, whether the intermediate tables are useful or not when
>MERGING some segments?
>
>2015-04-02 14:26 GMT+08:00 Shi, Shaofeng <sh...@ebay.com>:
>
>> Kylin creates an intermediate flat table when building a cube each time;
>> Due to some reason the table wasn¹t dropped automatically after cube be
>> built; So after some period of time, we suggest you to drop these
>>tables to
>> cleanup the HDFS space; Here is a simple command to drop all these temp
>> tables in batch (make sure there is no cube being built):
>>
>> hive -e 'show tables' | grep kylin_intermediate_ | xargs -I '{}' hive -e
>> 'drop table {}'
>>
>>
>>
Re: Command to drop all Kylin intermediate flat tables
Posted by dong wang <el...@gmail.com>.
for this, I have a question that after building the cube segment
successfully, whether the intermediate tables are useful or not when
MERGING some segments?
2015-04-02 14:26 GMT+08:00 Shi, Shaofeng <sh...@ebay.com>:
> Kylin creates an intermediate flat table when building a cube each time;
> Due to some reason the table wasn’t dropped automatically after cube be
> built; So after some period of time, we suggest you to drop these tables to
> cleanup the HDFS space; Here is a simple command to drop all these temp
> tables in batch (make sure there is no cube being built):
>
> hive -e 'show tables' | grep kylin_intermediate_ | xargs -I '{}' hive -e
> 'drop table {}'
>
>
>