You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by dong wang <el...@gmail.com> on 2015/12/11 12:09:18 UTC
about assigning different mr job queue to different user groups
inside one kylin instances
Currently, we have different business groups, we want to assign different
mr job queue to different businiess user groups inside ONLY ONE kylin
instance, do we have this feature? and as searched, I find the following
piece:
public static final String KYLIN_MAP_JOB_QUEUE = "mapred.job.queue.name";
If we can pass a parameter to indicate different users to use different mr
job queue when building their own jobs with updating the codes, a question
is that is there any potential problems to conduct the action?
Re: about assigning different mr job queue to different user groups
inside one kylin instances
Posted by dong wang <el...@gmail.com>.
I have added the logic(https://github.com/apache/kylin/pull/17)to auto
detect the kylin_job_conf_medium/small/large_projectname.xml first, then
kylin_job_conf_projectname.xml, then kylin_job_conf_medium/small/large.xml,
finally kylin_job_conf.xml, if still not found, throws an exception
2015-12-14 16:22 GMT+08:00 yu feng <ol...@gmail.com>:
> Maybe a quick solution is just create a config file for every project named
> ''kylin_job_conf_${projectname}.xml", it will just modify code that
> selecting config file for a MR job, and you can control every MR config
> property in project level.
>
> 2015-12-11 22:00 GMT+08:00 Shi, Shaofeng <sh...@ebay.com>:
>
> > This is a valid scenario; So far Kylin doesn¹t have project or cube level
> > job configurations, if it be implemented, you problem will be solved.
> >
> > On 12/11/15, 7:09 PM, "dong wang" <el...@gmail.com> wrote:
> >
> > >Currently, we have different business groups, we want to assign
> different
> > >mr job queue to different businiess user groups inside ONLY ONE kylin
> > >instance, do we have this feature? and as searched, I find the
> following
> > >piece:
> > >
> > >public static final String KYLIN_MAP_JOB_QUEUE = "mapred.job.queue.name
> ";
> > >
> > >
> > >If we can pass a parameter to indicate different users to use different
> mr
> > >job queue when building their own jobs with updating the codes, a
> question
> > >is that is there any potential problems to conduct the action?
> >
> >
>
Re: about assigning different mr job queue to different user groups
inside one kylin instances
Posted by "Shi, Shaofeng" <sh...@ebay.com>.
Thanks Dong; I put a comment in your JIRA:
https://issues.apache.org/jira/browse/KYLIN-1241, we can discuss there.
On 12/21/15, 10:29 AM, "dong wang" <el...@gmail.com> wrote:
>I have added the logic ( https://github.com/apache/kylin/pull/17 ) to auto
>detect the kylin_job_conf_medium/small/large_projectname.xml first, then
>kylin_job_conf_projectname.xml, then
>kylin_job_conf_medium/small/large.xml,
>finally kylin_job_conf.xml, if still not found, throws an exception
>
>2015-12-14 16:22 GMT+08:00 yu feng <ol...@gmail.com>:
>
>> Maybe a quick solution is just create a config file for every project
>>named
>> ''kylin_job_conf_${projectname}.xml", it will just modify code that
>> selecting config file for a MR job, and you can control every MR config
>> property in project level.
>>
>> 2015-12-11 22:00 GMT+08:00 Shi, Shaofeng <sh...@ebay.com>:
>>
>> > This is a valid scenario; So far Kylin doesn¹t have project or cube
>>level
>> > job configurations, if it be implemented, you problem will be solved.
>> >
>> > On 12/11/15, 7:09 PM, "dong wang" <el...@gmail.com> wrote:
>> >
>> > >Currently, we have different business groups, we want to assign
>> different
>> > >mr job queue to different businiess user groups inside ONLY ONE kylin
>> > >instance, do we have this feature? and as searched, I find the
>> following
>> > >piece:
>> > >
>> > >public static final String KYLIN_MAP_JOB_QUEUE =
>>"mapred.job.queue.name
>> ";
>> > >
>> > >
>> > >If we can pass a parameter to indicate different users to use
>>different
>> mr
>> > >job queue when building their own jobs with updating the codes, a
>> question
>> > >is that is there any potential problems to conduct the action?
>> >
>> >
>>
Re: about assigning different mr job queue to different user groups
inside one kylin instances
Posted by dong wang <el...@gmail.com>.
I have added the logic ( https://github.com/apache/kylin/pull/17 ) to auto
detect the kylin_job_conf_medium/small/large_projectname.xml first, then
kylin_job_conf_projectname.xml, then kylin_job_conf_medium/small/large.xml,
finally kylin_job_conf.xml, if still not found, throws an exception
2015-12-14 16:22 GMT+08:00 yu feng <ol...@gmail.com>:
> Maybe a quick solution is just create a config file for every project named
> ''kylin_job_conf_${projectname}.xml", it will just modify code that
> selecting config file for a MR job, and you can control every MR config
> property in project level.
>
> 2015-12-11 22:00 GMT+08:00 Shi, Shaofeng <sh...@ebay.com>:
>
> > This is a valid scenario; So far Kylin doesn¹t have project or cube level
> > job configurations, if it be implemented, you problem will be solved.
> >
> > On 12/11/15, 7:09 PM, "dong wang" <el...@gmail.com> wrote:
> >
> > >Currently, we have different business groups, we want to assign
> different
> > >mr job queue to different businiess user groups inside ONLY ONE kylin
> > >instance, do we have this feature? and as searched, I find the
> following
> > >piece:
> > >
> > >public static final String KYLIN_MAP_JOB_QUEUE = "mapred.job.queue.name
> ";
> > >
> > >
> > >If we can pass a parameter to indicate different users to use different
> mr
> > >job queue when building their own jobs with updating the codes, a
> question
> > >is that is there any potential problems to conduct the action?
> >
> >
>
Re: about assigning different mr job queue to different user groups
inside one kylin instances
Posted by yu feng <ol...@gmail.com>.
Maybe a quick solution is just create a config file for every project named
''kylin_job_conf_${projectname}.xml", it will just modify code that
selecting config file for a MR job, and you can control every MR config
property in project level.
2015-12-11 22:00 GMT+08:00 Shi, Shaofeng <sh...@ebay.com>:
> This is a valid scenario; So far Kylin doesn¹t have project or cube level
> job configurations, if it be implemented, you problem will be solved.
>
> On 12/11/15, 7:09 PM, "dong wang" <el...@gmail.com> wrote:
>
> >Currently, we have different business groups, we want to assign different
> >mr job queue to different businiess user groups inside ONLY ONE kylin
> >instance, do we have this feature? and as searched, I find the following
> >piece:
> >
> >public static final String KYLIN_MAP_JOB_QUEUE = "mapred.job.queue.name";
> >
> >
> >If we can pass a parameter to indicate different users to use different mr
> >job queue when building their own jobs with updating the codes, a question
> >is that is there any potential problems to conduct the action?
>
>
Re: about assigning different mr job queue to different user groups
inside one kylin instances
Posted by "Shi, Shaofeng" <sh...@ebay.com>.
This is a valid scenario; So far Kylin doesn¹t have project or cube level
job configurations, if it be implemented, you problem will be solved.
On 12/11/15, 7:09 PM, "dong wang" <el...@gmail.com> wrote:
>Currently, we have different business groups, we want to assign different
>mr job queue to different businiess user groups inside ONLY ONE kylin
>instance, do we have this feature? and as searched, I find the following
>piece:
>
>public static final String KYLIN_MAP_JOB_QUEUE = "mapred.job.queue.name";
>
>
>If we can pass a parameter to indicate different users to use different mr
>job queue when building their own jobs with updating the codes, a question
>is that is there any potential problems to conduct the action?