You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by Vineet Mishra <cl...@gmail.com> on 2015/06/12 09:05:30 UTC
Multiple Cubes for the Same Table
Hi All,
I was curious to know as when I am created multiple cubes for the same
table as source and later I am referring the cube to get the cube data.
Which table will it refer since our reference is still limited to source
table but the query is made from hbase table. So I wanted to have the
understanding as how/what changes need to be done so as to assure the
multiple cubes are formed and queries for the same table.
Urgent Call!
Thanks!
Re: Multiple Cubes for the Same Table
Posted by Vineet Mishra <cl...@gmail.com>.
Hi Shi,
Wanted to know how "Kylin will pick the ³best matched² one (with a set of
rules)" and moreover why does cube build job runs with single M/R job. Is
it really necessary to have single mapper and reducer job or it can be
tweaked as well, because the cube creation takes hell lot of time, around
2:30 hrs which is really too much in future when the data grows!
It there any work around to make the M/R to spawn multiple tasks to
expedite the running process?
Thanks!
On Fri, Jun 12, 2015 at 12:49 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
> Only 1 cube will be picked up to server a query; If you have multiple
> active cubes with the same source table, Kylin will pick the ³best
> matched² one (with a set of rules); You can get the info in kylin.logs;
> for example:
>
> QueryRouter.findBestMatchCube(QueryRouter.java:146)] - Find candidates by
> table Š cube1, cube2, ...
> Š. Exclude cube2 as ...
> findBestMatchCube(QueryRouter.java:185)] - Use weak matched cube cube1
>
> If you have preference on a certain cube, you can disable others, only
> leave the preferred as ³active², then it will be the only one to server
> SQL queries;
>
> On 6/12/15, 3:05 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
>
> >Hi All,
> >
> >I was curious to know as when I am created multiple cubes for the same
> >table as source and later I am referring the cube to get the cube data.
> >
> >Which table will it refer since our reference is still limited to source
> >table but the query is made from hbase table. So I wanted to have the
> >understanding as how/what changes need to be done so as to assure the
> >multiple cubes are formed and queries for the same table.
> >
> >Urgent Call!
> >
> >Thanks!
>
>
Re: Multiple Cubes for the Same Table
Posted by Vineet Mishra <cl...@gmail.com>.
That's Perfect! :)
Btw, Where does it store its Meta Info?
On Sun, Jun 14, 2015 at 7:41 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
> The preconditions is, the candidate cubes need match with SQL query,
> including join conditions (inner, left), dimension, measures; If more than
> 1 cubes match the condition, Kylin will select the most efficient one, for
> example: Cube A has 10 dimensions, Cube B has 20 dimensions, use A;
>
> On 6/14/15, 6:53 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
>
> >Hi Shi,
> >
> >Wanted to know how "Kylin will pick the ³best matched² one (with a set of
> >rules)" and moreover why does cube build job runs with single M/R job. Is
> >it really necessary to have single mapper and reducer job or it can be
> >tweaked as well, because the cube creation takes hell lot of time, around
> >2:30 hrs which is really too much in future when the data grows!
> >
> >It there any work around to make the M/R to spawn multiple tasks to
> >expedite the running process?
> >
> >Thanks!
> >
> >On Fri, Jun 12, 2015 at 12:49 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
> >
> >> Only 1 cube will be picked up to server a query; If you have multiple
> >> active cubes with the same source table, Kylin will pick the ³best
> >> matched² one (with a set of rules); You can get the info in kylin.logs;
> >> for example:
> >>
> >> QueryRouter.findBestMatchCube(QueryRouter.java:146)] - Find candidates
> >>by
> >> table Š cube1, cube2, ...
> >> Š. Exclude cube2 as ...
> >> findBestMatchCube(QueryRouter.java:185)] - Use weak matched cube cube1
> >>
> >> If you have preference on a certain cube, you can disable others, only
> >> leave the preferred as ³active², then it will be the only one to server
> >> SQL queries;
> >>
> >> On 6/12/15, 3:05 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
> >>
> >> >Hi All,
> >> >
> >> >I was curious to know as when I am created multiple cubes for the same
> >> >table as source and later I am referring the cube to get the cube data.
> >> >
> >> >Which table will it refer since our reference is still limited to
> >>source
> >> >table but the query is made from hbase table. So I wanted to have the
> >> >understanding as how/what changes need to be done so as to assure the
> >> >multiple cubes are formed and queries for the same table.
> >> >
> >> >Urgent Call!
> >> >
> >> >Thanks!
> >>
> >>
>
>
Re: Multiple Cubes for the Same Table
Posted by "Shi, Shaofeng" <sh...@ebay.com>.
The preconditions is, the candidate cubes need match with SQL query,
including join conditions (inner, left), dimension, measures; If more than
1 cubes match the condition, Kylin will select the most efficient one, for
example: Cube A has 10 dimensions, Cube B has 20 dimensions, use A;
On 6/14/15, 6:53 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
>Hi Shi,
>
>Wanted to know how "Kylin will pick the ³best matched² one (with a set of
>rules)" and moreover why does cube build job runs with single M/R job. Is
>it really necessary to have single mapper and reducer job or it can be
>tweaked as well, because the cube creation takes hell lot of time, around
>2:30 hrs which is really too much in future when the data grows!
>
>It there any work around to make the M/R to spawn multiple tasks to
>expedite the running process?
>
>Thanks!
>
>On Fri, Jun 12, 2015 at 12:49 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
>
>> Only 1 cube will be picked up to server a query; If you have multiple
>> active cubes with the same source table, Kylin will pick the ³best
>> matched² one (with a set of rules); You can get the info in kylin.logs;
>> for example:
>>
>> QueryRouter.findBestMatchCube(QueryRouter.java:146)] - Find candidates
>>by
>> table Š cube1, cube2, ...
>> Š. Exclude cube2 as ...
>> findBestMatchCube(QueryRouter.java:185)] - Use weak matched cube cube1
>>
>> If you have preference on a certain cube, you can disable others, only
>> leave the preferred as ³active², then it will be the only one to server
>> SQL queries;
>>
>> On 6/12/15, 3:05 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
>>
>> >Hi All,
>> >
>> >I was curious to know as when I am created multiple cubes for the same
>> >table as source and later I am referring the cube to get the cube data.
>> >
>> >Which table will it refer since our reference is still limited to
>>source
>> >table but the query is made from hbase table. So I wanted to have the
>> >understanding as how/what changes need to be done so as to assure the
>> >multiple cubes are formed and queries for the same table.
>> >
>> >Urgent Call!
>> >
>> >Thanks!
>>
>>
Re: Multiple Cubes for the Same Table
Posted by Vineet Mishra <cl...@gmail.com>.
Hi Shi,
Wanted to know how "Kylin will pick the ³best matched² one (with a set of
rules)" and moreover why does cube build job runs with single M/R job. Is
it really necessary to have single mapper and reducer job or it can be
tweaked as well, because the cube creation takes hell lot of time, around
2:30 hrs which is really too much in future when the data grows!
It there any work around to make the M/R to spawn multiple tasks to
expedite the running process?
Thanks!
On Fri, Jun 12, 2015 at 12:49 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
> Only 1 cube will be picked up to server a query; If you have multiple
> active cubes with the same source table, Kylin will pick the ³best
> matched² one (with a set of rules); You can get the info in kylin.logs;
> for example:
>
> QueryRouter.findBestMatchCube(QueryRouter.java:146)] - Find candidates by
> table Š cube1, cube2, ...
> Š. Exclude cube2 as ...
> findBestMatchCube(QueryRouter.java:185)] - Use weak matched cube cube1
>
> If you have preference on a certain cube, you can disable others, only
> leave the preferred as ³active², then it will be the only one to server
> SQL queries;
>
> On 6/12/15, 3:05 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
>
> >Hi All,
> >
> >I was curious to know as when I am created multiple cubes for the same
> >table as source and later I am referring the cube to get the cube data.
> >
> >Which table will it refer since our reference is still limited to source
> >table but the query is made from hbase table. So I wanted to have the
> >understanding as how/what changes need to be done so as to assure the
> >multiple cubes are formed and queries for the same table.
> >
> >Urgent Call!
> >
> >Thanks!
>
>
Re: Multiple Cubes for the Same Table
Posted by "Shi, Shaofeng" <sh...@ebay.com>.
Only 1 cube will be picked up to server a query; If you have multiple
active cubes with the same source table, Kylin will pick the ³best
matched² one (with a set of rules); You can get the info in kylin.logs;
for example:
QueryRouter.findBestMatchCube(QueryRouter.java:146)] - Find candidates by
table Š cube1, cube2, ...
Š. Exclude cube2 as ...
findBestMatchCube(QueryRouter.java:185)] - Use weak matched cube cube1
If you have preference on a certain cube, you can disable others, only
leave the preferred as ³active², then it will be the only one to server
SQL queries;
On 6/12/15, 3:05 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
>Hi All,
>
>I was curious to know as when I am created multiple cubes for the same
>table as source and later I am referring the cube to get the cube data.
>
>Which table will it refer since our reference is still limited to source
>table but the query is made from hbase table. So I wanted to have the
>understanding as how/what changes need to be done so as to assure the
>multiple cubes are formed and queries for the same table.
>
>Urgent Call!
>
>Thanks!