You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Bertrand Dechoux <de...@gmail.com> on 2012/09/13 14:16:58 UTC
Configuration of hadoop.job.history.user.location on a user basis
Hi,
I would like to be able to fully isolate the quota of each user.
That is, I would like to put everything related to a user into her/his home
directory /user/<user>.
Among other points of configuration, I would like to be able to configure
hadoop.job.history.user.location.
But I can't see how it could be done.
How can I have a different path for each user?
How do I make sure that each job will still have its own directory? (Even
when no output dir is defined for the job.)
Other question, slightly related. If I deactivate those logs using the
"none" configuration, it should only impact the copy of the user, should it
not?
The user should still be able to browse the logs using the version provided
through the jobtracker web interface, should it not?
Regards
Bertrand
Re: Configuration of hadoop.job.history.user.location on a user basis
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi Hemanth,
I thought of that but I didn't test it because I need the same for the job
id. Of course, I could generate my own id but I don't like that solution. I
will look at hadoop.tmp.dir to see if it can solve my issue differently.
Regards
Bertrand
On Fri, Sep 14, 2012 at 5:54 AM, Hemanth Yamijala <yhemanth@thoughtworks.com
> wrote:
> Hi Bretrand,
>
> Could you try using user.name as a property in the value of the config
> item ? Something like /user/${user.name}/history ?
>
> Thanks
> hemanth
>
>
> On Thu, Sep 13, 2012 at 5:46 PM, Bertrand Dechoux <de...@gmail.com>wrote:
>
>> Hi,
>>
>> I would like to be able to fully isolate the quota of each user.
>> That is, I would like to put everything related to a user into her/his
>> home directory /user/<user>.
>>
>> Among other points of configuration, I would like to be able to configure
>> hadoop.job.history.user.location.
>> But I can't see how it could be done.
>>
>> How can I have a different path for each user?
>> How do I make sure that each job will still have its own directory? (Even
>> when no output dir is defined for the job.)
>>
>> Other question, slightly related. If I deactivate those logs using the
>> "none" configuration, it should only impact the copy of the user, should it
>> not?
>> The user should still be able to browse the logs using the version
>> provided through the jobtracker web interface, should it not?
>>
>> Regards
>>
>> Bertrand
>>
>
>
--
Bertrand Dechoux
Re: Configuration of hadoop.job.history.user.location on a user basis
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi Hemanth,
I thought of that but I didn't test it because I need the same for the job
id. Of course, I could generate my own id but I don't like that solution. I
will look at hadoop.tmp.dir to see if it can solve my issue differently.
Regards
Bertrand
On Fri, Sep 14, 2012 at 5:54 AM, Hemanth Yamijala <yhemanth@thoughtworks.com
> wrote:
> Hi Bretrand,
>
> Could you try using user.name as a property in the value of the config
> item ? Something like /user/${user.name}/history ?
>
> Thanks
> hemanth
>
>
> On Thu, Sep 13, 2012 at 5:46 PM, Bertrand Dechoux <de...@gmail.com>wrote:
>
>> Hi,
>>
>> I would like to be able to fully isolate the quota of each user.
>> That is, I would like to put everything related to a user into her/his
>> home directory /user/<user>.
>>
>> Among other points of configuration, I would like to be able to configure
>> hadoop.job.history.user.location.
>> But I can't see how it could be done.
>>
>> How can I have a different path for each user?
>> How do I make sure that each job will still have its own directory? (Even
>> when no output dir is defined for the job.)
>>
>> Other question, slightly related. If I deactivate those logs using the
>> "none" configuration, it should only impact the copy of the user, should it
>> not?
>> The user should still be able to browse the logs using the version
>> provided through the jobtracker web interface, should it not?
>>
>> Regards
>>
>> Bertrand
>>
>
>
--
Bertrand Dechoux
Re: Configuration of hadoop.job.history.user.location on a user basis
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi Hemanth,
I thought of that but I didn't test it because I need the same for the job
id. Of course, I could generate my own id but I don't like that solution. I
will look at hadoop.tmp.dir to see if it can solve my issue differently.
Regards
Bertrand
On Fri, Sep 14, 2012 at 5:54 AM, Hemanth Yamijala <yhemanth@thoughtworks.com
> wrote:
> Hi Bretrand,
>
> Could you try using user.name as a property in the value of the config
> item ? Something like /user/${user.name}/history ?
>
> Thanks
> hemanth
>
>
> On Thu, Sep 13, 2012 at 5:46 PM, Bertrand Dechoux <de...@gmail.com>wrote:
>
>> Hi,
>>
>> I would like to be able to fully isolate the quota of each user.
>> That is, I would like to put everything related to a user into her/his
>> home directory /user/<user>.
>>
>> Among other points of configuration, I would like to be able to configure
>> hadoop.job.history.user.location.
>> But I can't see how it could be done.
>>
>> How can I have a different path for each user?
>> How do I make sure that each job will still have its own directory? (Even
>> when no output dir is defined for the job.)
>>
>> Other question, slightly related. If I deactivate those logs using the
>> "none" configuration, it should only impact the copy of the user, should it
>> not?
>> The user should still be able to browse the logs using the version
>> provided through the jobtracker web interface, should it not?
>>
>> Regards
>>
>> Bertrand
>>
>
>
--
Bertrand Dechoux
Re: Configuration of hadoop.job.history.user.location on a user basis
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi Hemanth,
I thought of that but I didn't test it because I need the same for the job
id. Of course, I could generate my own id but I don't like that solution. I
will look at hadoop.tmp.dir to see if it can solve my issue differently.
Regards
Bertrand
On Fri, Sep 14, 2012 at 5:54 AM, Hemanth Yamijala <yhemanth@thoughtworks.com
> wrote:
> Hi Bretrand,
>
> Could you try using user.name as a property in the value of the config
> item ? Something like /user/${user.name}/history ?
>
> Thanks
> hemanth
>
>
> On Thu, Sep 13, 2012 at 5:46 PM, Bertrand Dechoux <de...@gmail.com>wrote:
>
>> Hi,
>>
>> I would like to be able to fully isolate the quota of each user.
>> That is, I would like to put everything related to a user into her/his
>> home directory /user/<user>.
>>
>> Among other points of configuration, I would like to be able to configure
>> hadoop.job.history.user.location.
>> But I can't see how it could be done.
>>
>> How can I have a different path for each user?
>> How do I make sure that each job will still have its own directory? (Even
>> when no output dir is defined for the job.)
>>
>> Other question, slightly related. If I deactivate those logs using the
>> "none" configuration, it should only impact the copy of the user, should it
>> not?
>> The user should still be able to browse the logs using the version
>> provided through the jobtracker web interface, should it not?
>>
>> Regards
>>
>> Bertrand
>>
>
>
--
Bertrand Dechoux
Re: Configuration of hadoop.job.history.user.location on a user basis
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
Hi Bretrand,
Could you try using user.name as a property in the value of the config item
? Something like /user/${user.name}/history ?
Thanks
hemanth
On Thu, Sep 13, 2012 at 5:46 PM, Bertrand Dechoux <de...@gmail.com>wrote:
> Hi,
>
> I would like to be able to fully isolate the quota of each user.
> That is, I would like to put everything related to a user into her/his
> home directory /user/<user>.
>
> Among other points of configuration, I would like to be able to configure
> hadoop.job.history.user.location.
> But I can't see how it could be done.
>
> How can I have a different path for each user?
> How do I make sure that each job will still have its own directory? (Even
> when no output dir is defined for the job.)
>
> Other question, slightly related. If I deactivate those logs using the
> "none" configuration, it should only impact the copy of the user, should it
> not?
> The user should still be able to browse the logs using the version
> provided through the jobtracker web interface, should it not?
>
> Regards
>
> Bertrand
>
Re: Configuration of hadoop.job.history.user.location on a user basis
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
Hi Bretrand,
Could you try using user.name as a property in the value of the config item
? Something like /user/${user.name}/history ?
Thanks
hemanth
On Thu, Sep 13, 2012 at 5:46 PM, Bertrand Dechoux <de...@gmail.com>wrote:
> Hi,
>
> I would like to be able to fully isolate the quota of each user.
> That is, I would like to put everything related to a user into her/his
> home directory /user/<user>.
>
> Among other points of configuration, I would like to be able to configure
> hadoop.job.history.user.location.
> But I can't see how it could be done.
>
> How can I have a different path for each user?
> How do I make sure that each job will still have its own directory? (Even
> when no output dir is defined for the job.)
>
> Other question, slightly related. If I deactivate those logs using the
> "none" configuration, it should only impact the copy of the user, should it
> not?
> The user should still be able to browse the logs using the version
> provided through the jobtracker web interface, should it not?
>
> Regards
>
> Bertrand
>
Re: Configuration of hadoop.job.history.user.location on a user basis
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
Hi Bretrand,
Could you try using user.name as a property in the value of the config item
? Something like /user/${user.name}/history ?
Thanks
hemanth
On Thu, Sep 13, 2012 at 5:46 PM, Bertrand Dechoux <de...@gmail.com>wrote:
> Hi,
>
> I would like to be able to fully isolate the quota of each user.
> That is, I would like to put everything related to a user into her/his
> home directory /user/<user>.
>
> Among other points of configuration, I would like to be able to configure
> hadoop.job.history.user.location.
> But I can't see how it could be done.
>
> How can I have a different path for each user?
> How do I make sure that each job will still have its own directory? (Even
> when no output dir is defined for the job.)
>
> Other question, slightly related. If I deactivate those logs using the
> "none" configuration, it should only impact the copy of the user, should it
> not?
> The user should still be able to browse the logs using the version
> provided through the jobtracker web interface, should it not?
>
> Regards
>
> Bertrand
>
Re: Configuration of hadoop.job.history.user.location on a user basis
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
Hi Bretrand,
Could you try using user.name as a property in the value of the config item
? Something like /user/${user.name}/history ?
Thanks
hemanth
On Thu, Sep 13, 2012 at 5:46 PM, Bertrand Dechoux <de...@gmail.com>wrote:
> Hi,
>
> I would like to be able to fully isolate the quota of each user.
> That is, I would like to put everything related to a user into her/his
> home directory /user/<user>.
>
> Among other points of configuration, I would like to be able to configure
> hadoop.job.history.user.location.
> But I can't see how it could be done.
>
> How can I have a different path for each user?
> How do I make sure that each job will still have its own directory? (Even
> when no output dir is defined for the job.)
>
> Other question, slightly related. If I deactivate those logs using the
> "none" configuration, it should only impact the copy of the user, should it
> not?
> The user should still be able to browse the logs using the version
> provided through the jobtracker web interface, should it not?
>
> Regards
>
> Bertrand
>