You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by John Lilley <jo...@redpoint.net> on 2013/06/06 21:44:55 UTC
log4j defaults for AM & task
In debugging a custom AM and its tasks, I want to create log4j settings so that the log output goes to someplace "standard" where I can collect all of the logs at the end of the run and see what happened to each part. Is there a convention for doing this? Does each task get a well-defined and unique directory where I can leave log files at the end of the run?
Thanks,
john
RE: log4j defaults for AM & task
Posted by John Lilley <jo...@redpoint.net>.
Thanks I would have never guessed ;-)
I assume it will sub anywhere on the cmdline like:
mycommand -logdir=<LOG_DIR>
john
-----Original Message-----
From: Arun C Murthy [mailto:acm@hortonworks.com]
Sent: Thursday, June 06, 2013 2:25 PM
To: user@hadoop.apache.org
Subject: Re: log4j defaults for AM & task
Unfortunately this isnt' well documented, but if the cmd-line for your container has <LOG_DIR>, that variable will be substituted by the NodeManager with the directory where the logs will be put.
For e.g. you can pass in: myapp.py 2> <LOG_DIR>/stderr etc.
I've opened https://issues.apache.org/jira/browse/YARN-772 to track documentation for this and other such features which are available.
thanks,
Arun
On Jun 6, 2013, at 12:44 PM, John Lilley wrote:
> In debugging a custom AM and its tasks, I want to create log4j settings so that the log output goes to someplace "standard" where I can collect all of the logs at the end of the run and see what happened to each part. Is there a convention for doing this? Does each task get a well-defined and unique directory where I can leave log files at the end of the run?
> Thanks,
> john
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
RE: log4j defaults for AM & task
Posted by John Lilley <jo...@redpoint.net>.
Thanks I would have never guessed ;-)
I assume it will sub anywhere on the cmdline like:
mycommand -logdir=<LOG_DIR>
john
-----Original Message-----
From: Arun C Murthy [mailto:acm@hortonworks.com]
Sent: Thursday, June 06, 2013 2:25 PM
To: user@hadoop.apache.org
Subject: Re: log4j defaults for AM & task
Unfortunately this isnt' well documented, but if the cmd-line for your container has <LOG_DIR>, that variable will be substituted by the NodeManager with the directory where the logs will be put.
For e.g. you can pass in: myapp.py 2> <LOG_DIR>/stderr etc.
I've opened https://issues.apache.org/jira/browse/YARN-772 to track documentation for this and other such features which are available.
thanks,
Arun
On Jun 6, 2013, at 12:44 PM, John Lilley wrote:
> In debugging a custom AM and its tasks, I want to create log4j settings so that the log output goes to someplace "standard" where I can collect all of the logs at the end of the run and see what happened to each part. Is there a convention for doing this? Does each task get a well-defined and unique directory where I can leave log files at the end of the run?
> Thanks,
> john
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
RE: log4j defaults for AM & task
Posted by John Lilley <jo...@redpoint.net>.
Thanks I would have never guessed ;-)
I assume it will sub anywhere on the cmdline like:
mycommand -logdir=<LOG_DIR>
john
-----Original Message-----
From: Arun C Murthy [mailto:acm@hortonworks.com]
Sent: Thursday, June 06, 2013 2:25 PM
To: user@hadoop.apache.org
Subject: Re: log4j defaults for AM & task
Unfortunately this isnt' well documented, but if the cmd-line for your container has <LOG_DIR>, that variable will be substituted by the NodeManager with the directory where the logs will be put.
For e.g. you can pass in: myapp.py 2> <LOG_DIR>/stderr etc.
I've opened https://issues.apache.org/jira/browse/YARN-772 to track documentation for this and other such features which are available.
thanks,
Arun
On Jun 6, 2013, at 12:44 PM, John Lilley wrote:
> In debugging a custom AM and its tasks, I want to create log4j settings so that the log output goes to someplace "standard" where I can collect all of the logs at the end of the run and see what happened to each part. Is there a convention for doing this? Does each task get a well-defined and unique directory where I can leave log files at the end of the run?
> Thanks,
> john
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
RE: log4j defaults for AM & task
Posted by John Lilley <jo...@redpoint.net>.
Thanks I would have never guessed ;-)
I assume it will sub anywhere on the cmdline like:
mycommand -logdir=<LOG_DIR>
john
-----Original Message-----
From: Arun C Murthy [mailto:acm@hortonworks.com]
Sent: Thursday, June 06, 2013 2:25 PM
To: user@hadoop.apache.org
Subject: Re: log4j defaults for AM & task
Unfortunately this isnt' well documented, but if the cmd-line for your container has <LOG_DIR>, that variable will be substituted by the NodeManager with the directory where the logs will be put.
For e.g. you can pass in: myapp.py 2> <LOG_DIR>/stderr etc.
I've opened https://issues.apache.org/jira/browse/YARN-772 to track documentation for this and other such features which are available.
thanks,
Arun
On Jun 6, 2013, at 12:44 PM, John Lilley wrote:
> In debugging a custom AM and its tasks, I want to create log4j settings so that the log output goes to someplace "standard" where I can collect all of the logs at the end of the run and see what happened to each part. Is there a convention for doing this? Does each task get a well-defined and unique directory where I can leave log files at the end of the run?
> Thanks,
> john
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: log4j defaults for AM & task
Posted by Arun C Murthy <ac...@hortonworks.com>.
Unfortunately this isnt' well documented, but if the cmd-line for your container has <LOG_DIR>, that variable will be substituted by the NodeManager with the directory where the logs will be put.
For e.g. you can pass in: myapp.py 2> <LOG_DIR>/stderr etc.
I've opened https://issues.apache.org/jira/browse/YARN-772 to track documentation for this and other such features which are available.
thanks,
Arun
On Jun 6, 2013, at 12:44 PM, John Lilley wrote:
> In debugging a custom AM and its tasks, I want to create log4j settings so that the log output goes to someplace “standard” where I can collect all of the logs at the end of the run and see what happened to each part. Is there a convention for doing this? Does each task get a well-defined and unique directory where I can leave log files at the end of the run?
> Thanks,
> john
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: log4j defaults for AM & task
Posted by Arun C Murthy <ac...@hortonworks.com>.
Unfortunately this isnt' well documented, but if the cmd-line for your container has <LOG_DIR>, that variable will be substituted by the NodeManager with the directory where the logs will be put.
For e.g. you can pass in: myapp.py 2> <LOG_DIR>/stderr etc.
I've opened https://issues.apache.org/jira/browse/YARN-772 to track documentation for this and other such features which are available.
thanks,
Arun
On Jun 6, 2013, at 12:44 PM, John Lilley wrote:
> In debugging a custom AM and its tasks, I want to create log4j settings so that the log output goes to someplace “standard” where I can collect all of the logs at the end of the run and see what happened to each part. Is there a convention for doing this? Does each task get a well-defined and unique directory where I can leave log files at the end of the run?
> Thanks,
> john
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: log4j defaults for AM & task
Posted by Arun C Murthy <ac...@hortonworks.com>.
Unfortunately this isnt' well documented, but if the cmd-line for your container has <LOG_DIR>, that variable will be substituted by the NodeManager with the directory where the logs will be put.
For e.g. you can pass in: myapp.py 2> <LOG_DIR>/stderr etc.
I've opened https://issues.apache.org/jira/browse/YARN-772 to track documentation for this and other such features which are available.
thanks,
Arun
On Jun 6, 2013, at 12:44 PM, John Lilley wrote:
> In debugging a custom AM and its tasks, I want to create log4j settings so that the log output goes to someplace “standard” where I can collect all of the logs at the end of the run and see what happened to each part. Is there a convention for doing this? Does each task get a well-defined and unique directory where I can leave log files at the end of the run?
> Thanks,
> john
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: log4j defaults for AM & task
Posted by Arun C Murthy <ac...@hortonworks.com>.
Unfortunately this isnt' well documented, but if the cmd-line for your container has <LOG_DIR>, that variable will be substituted by the NodeManager with the directory where the logs will be put.
For e.g. you can pass in: myapp.py 2> <LOG_DIR>/stderr etc.
I've opened https://issues.apache.org/jira/browse/YARN-772 to track documentation for this and other such features which are available.
thanks,
Arun
On Jun 6, 2013, at 12:44 PM, John Lilley wrote:
> In debugging a custom AM and its tasks, I want to create log4j settings so that the log output goes to someplace “standard” where I can collect all of the logs at the end of the run and see what happened to each part. Is there a convention for doing this? Does each task get a well-defined and unique directory where I can leave log files at the end of the run?
> Thanks,
> john
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/