about the metadata in the HBASE

[dong wang <el...@gmail.com>]

in kylin-0.71, how can we control to completely re-construct all the
metadata in HBASE(you can think it as truncate all the data of meta tables
in the HBASE), currently, it seems to be no effects when re-run the
script/package.sh

in my vague memory, is it right that the metadata will be re-constructed in
the HBASE?

Re: about the metadata in the HBASE

["Han, Luke" <lu...@ebay.com>]

Correction: 0.7.x will require 3 metadata tables:
kylin_metadata
kylin_metadata_acl
kylin_metadata_user




Thanks

Luke



On 3/6/15, 17:53, "hongbin ma" <ma...@apache.org> wrote:

>On Fri, Mar 6, 2015 at 5:36 PM, dong wang <el...@gmail.com> wrote:
>
>> kylin_metadata
>> kylin_metadata_acl
>> kylin_metadata_qa
>> kylin_metadata_qa_acl
>> kylin_metadata_qa_cube
>> kylin_metadata_qa_dict
>> kylin_metadata_qa_invertedindex
>> kylin_metadata_qa_job
>> kylin_metadata_qa_job_output
>> kylin_metadata_qa_proj
>> kylin_metadata_qa_table_snapshot
>> kylin_metadata_qa_user
>> kylin_metadata_user
>>
>
>​Upon starting, Kylin will check if these metadata tables exists, if yes,
>it will reuse them, otherwise it will create them.
>In 0.6 versions, metadata are spread in multiple tables categoried by
>modules, that why you're seeing:
>
>kylin_metadata_qa_acl
>kylin_metadata_qa_cube
>kylin_metadata_qa_dict
>kylin_metadata_qa_invertedindex
>kylin_metadata_qa_job
>kylin_metadata_qa_job_output
>kylin_metadata_qa_proj
>kylin_metadata_qa_table_snapshot
>kylin_metadata_qa_user
>
>In 0.7 versions, we will only use an "all in one" table, so if you're
>using
>0.7 versions, "kylin_metadata_qa" will replace all the above tables
>starting with "kylin_metadata_qa".
>
>Check out code:
>https://github.com/KylinOLAP/Kylin/blob/a3b4bccd90fe0852a93dd91f68cb37f08f
>8719f7/common/src/main/java/org/apache/kylin/common/persistence/HBaseResou
>rceStore.java#L90

Re: about the metadata in the HBASE

[hongbin ma <ma...@apache.org>]

On Fri, Mar 6, 2015 at 5:36 PM, dong wang <el...@gmail.com> wrote:

> kylin_metadata
> kylin_metadata_acl
> kylin_metadata_qa
> kylin_metadata_qa_acl
> kylin_metadata_qa_cube
> kylin_metadata_qa_dict
> kylin_metadata_qa_invertedindex
> kylin_metadata_qa_job
> kylin_metadata_qa_job_output
> kylin_metadata_qa_proj
> kylin_metadata_qa_table_snapshot
> kylin_metadata_qa_user
> kylin_metadata_user
>

​Upon starting, Kylin will check if these metadata tables exists, if yes,
it will reuse them, otherwise it will create them.
In 0.6 versions, metadata are spread in multiple tables categoried by
modules, that why you're seeing:

kylin_metadata_qa_acl
kylin_metadata_qa_cube
kylin_metadata_qa_dict
kylin_metadata_qa_invertedindex
kylin_metadata_qa_job
kylin_metadata_qa_job_output
kylin_metadata_qa_proj
kylin_metadata_qa_table_snapshot
kylin_metadata_qa_user

In 0.7 versions, we will only use an "all in one" table, so if you're using
0.7 versions, "kylin_metadata_qa" will replace all the above tables
starting with "kylin_metadata_qa".

Check out code:
https://github.com/KylinOLAP/Kylin/blob/a3b4bccd90fe0852a93dd91f68cb37f08f8719f7/common/src/main/java/org/apache/kylin/common/persistence/HBaseResourceStore.java#L90

Re: about the metadata in the HBASE

[dong wang <el...@gmail.com>]

actually, I don't care about the sample, I just want to know when the
following tables will be re-create and how to control the process?

kylin_metadata
kylin_metadata_acl
kylin_metadata_qa
kylin_metadata_qa_acl
kylin_metadata_qa_cube
kylin_metadata_qa_dict
kylin_metadata_qa_invertedindex
kylin_metadata_qa_job
kylin_metadata_qa_job_output
kylin_metadata_qa_proj
kylin_metadata_qa_table_snapshot
kylin_metadata_qa_user
kylin_metadata_user

2015-03-06 17:32 GMT+08:00 hongbin ma <ma...@apache.org>:

> Correct me if i get it wrong, are you seeking for preparing those sample
> cubes in your empty metadata store?
>
> On Fri, Mar 6, 2015 at 5:29 PM, dong wang <el...@gmail.com> wrote:
>
> > in my vague memory, is it right that the metadata will be re-constructed
> in
> > the HBASE in kylin-0.6x ?
> >
> > 2015-03-06 17:24 GMT+08:00 dong wang <el...@gmail.com>:
> >
> > > in kylin-0.71, how can we control to completely re-construct all the
> > > metadata in HBASE(you can think it as truncate all the data of meta
> > tables
> > > in the HBASE), currently, it seems to be no effects when re-run the
> > > script/package.sh
> > >
> > > in my vague memory, is it right that the metadata will be
> re-constructed
> > > in the HBASE?
> > >
> >
>

Re: about the metadata in the HBASE

[hongbin ma <ma...@apache.org>]

Correct me if i get it wrong, are you seeking for preparing those sample
cubes in your empty metadata store?

On Fri, Mar 6, 2015 at 5:29 PM, dong wang <el...@gmail.com> wrote:

> in my vague memory, is it right that the metadata will be re-constructed in
> the HBASE in kylin-0.6x ?
>
> 2015-03-06 17:24 GMT+08:00 dong wang <el...@gmail.com>:
>
> > in kylin-0.71, how can we control to completely re-construct all the
> > metadata in HBASE(you can think it as truncate all the data of meta
> tables
> > in the HBASE), currently, it seems to be no effects when re-run the
> > script/package.sh
> >
> > in my vague memory, is it right that the metadata will be re-constructed
> > in the HBASE?
> >
>

Re: about the metadata in the HBASE

[dong wang <el...@gmail.com>]

in my vague memory, is it right that the metadata will be re-constructed in
the HBASE in kylin-0.6x ?

2015-03-06 17:24 GMT+08:00 dong wang <el...@gmail.com>:

> in kylin-0.71, how can we control to completely re-construct all the
> metadata in HBASE(you can think it as truncate all the data of meta tables
> in the HBASE), currently, it seems to be no effects when re-run the
> script/package.sh
>
> in my vague memory, is it right that the metadata will be re-constructed
> in the HBASE?
>

Re: about the metadata in the HBASE

[hongbin ma <ma...@apache.org>]

​package.sh will only build artifacts. it will have no effect on kylin
metadata store in hbase.
what exactly do you want to do? do you want to clean up metadata store or
remove everything from the metadata store?
​