You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Lincoln Ritter <li...@gmail.com> on 2008/06/25 20:11:30 UTC
Job History Logging Location
Greetings,
I'm trying to get a handle on job history logging. According to the
documentation in 'hadoop-defaul.xml' the
'hadoop.job.history.user.location' determines where job history logs
are written. If not specified these logs go into
'<output-dir>/_logs/history'. This can cause problems with
applications that don't know about this convention. It would also be
nicer in my opinion to keep logs and data separate.
It seems to me that a nice way to handle this would be to put logs in
'/logs/<job-id>/history' or something.
Can this be done? Is there a need for the "job-id" folder? If this
can't be done, are there alternatives that work well.
-lincoln
--
lincolnritter.com
Job History Logging Location
Posted by Lincoln Ritter <li...@gmail.com>.
Greetings,
I'm trying to get a handle on job history logging. According to the
documentation in 'hadoop-defaul.xml' the
'hadoop.job.history.user.location' determines where job history logs
are written. If not specified these logs go into
'<output-dir>/_logs/history'. This can cause problems with
applications that don't know about this convention. It would also be
nicer in my opinion to keep logs and data separate.
It seems to me that a nice way to handle this would be to put logs in
'/logs/<job-id>/history' or something.
Can this be done? Is there a need for the "job-id" folder? If this
can't be done, are there alternatives that work well.
-lincoln
--
lincolnritter.com
Re: Job History Logging Location
Posted by Lincoln Ritter <li...@gmail.com>.
Hello again.
I answered my own question.
Setting 'hadoop.job.history.user.location' to 'logs' works fine.
Thanks anyway!
-lincoln
--
lincolnritter.com
On Wed, Jun 25, 2008 at 11:11 AM, Lincoln Ritter
<li...@gmail.com> wrote:
> Greetings,
>
> I'm trying to get a handle on job history logging. According to the
> documentation in 'hadoop-defaul.xml' the
> 'hadoop.job.history.user.location' determines where job history logs
> are written. If not specified these logs go into
> '<output-dir>/_logs/history'. This can cause problems with
> applications that don't know about this convention. It would also be
> nicer in my opinion to keep logs and data separate.
>
> It seems to me that a nice way to handle this would be to put logs in
> '/logs/<job-id>/history' or something.
>
> Can this be done? Is there a need for the "job-id" folder? If this
> can't be done, are there alternatives that work well.
>
> -lincoln
>
> --
> lincolnritter.com
>