You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Chris Schneider (JIRA)" <ji...@apache.org> on 2006/08/01 18:24:13 UTC
[jira] Created: (HADOOP-406) Tasks launched by tasktracker in
separate JVM can't generate log output
Tasks launched by tasktracker in separate JVM can't generate log output
-----------------------------------------------------------------------
Key: HADOOP-406
URL: http://issues.apache.org/jira/browse/HADOOP-406
Project: Hadoop
Issue Type: Bug
Components: mapred
Affects Versions: 0.4.0
Reporter: Chris Schneider
Child JVM's don't have access to logging config system properties. When the child JVM gets launched, it doesn't inherit the Java system properties hadoop.log.dir and hadoop.log.file (which are actually based on the Bash environment variables $HADOOP_LOG_DIR and $HADOOP_LOGFILE). This means that you get no log messages from the actual map/reduce tasks that are executing.
Stefan Groschupf reported this problem a while back:
-------------------------------------------------------------------------
To: hadoop-dev@lucene.apache.org
From: Stefan Groschupf <sg...@media-style.com>
Subject: tasks can't log bug?
Date: Tue, 25 Jul 2006 19:26:17 -0700
X-Virus-Checked: Checked by ClamAV on apache.org
Hi Hadoop developers,
I'm confused about the way logging works within map or reduce tasks.
Since tasks are launched in a new JVM the java system properties "hadoop.log.dir" and "hadoop.log.file" are not passed to the new JVM.
This prevents the child process from logging properly. In fact you get:
java.io.FileNotFoundException: / (Is a directory)
at java.io.FileOutputStream.openAppend(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:177)
at java.io.FileOutputStream.<init>(FileOutputStream.java:102)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:289)
at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:165)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:163)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:256)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:132)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:96)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:654)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:612)
at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.j
2006-07-25 15:59:07,553 INFO mapred.TaskTracker (TaskTracker.java:main(993)) - Child
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:415)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:441)
at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:4
at org.apache.log4j.LogManager.<clinit>(LogManager.java:122)
at org.apache.log4j.Logger.getLogger(Logger.java:104)
at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229)
at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImp
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAcc
at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529
at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235
at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:370)
at org.apache.hadoop.mapred.TaskTracker.<clinit>(TaskTracker.java:44)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:993)
We see several ways to solve this problem. First retrieve the properties "hadoop.log.dir" and "hadoop.log.file" from the mother JVM and then pass them to the child JVM as within the args parameter.
Second would be to access the environment variables "$HADOOP_LOG_DIR" and "$HADOOP_LOGFILE" using System.getEnv (java 1.5).
Third there would be a more general solution. Taskrunner would resolve any environment variables it found in "mapred.child.java.opts" by lookup the value using System.getEnv().
Eg:
unix:
export MAX_MEMORY = 200
hadoop-site.xml:
<name>mapred.child.java.opts</name>
<value>-Xmx${MAX_MEMORY}</value>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-406) Tasks launched by tasktracker in
separate JVM can't generate log output
Posted by "Doug Cook (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-406?page=comments#action_12431080 ]
Doug Cook commented on HADOOP-406:
----------------------------------
Hi, Doug.
The task trackers were definitely not logging the fetcher output until I applied and enabled this patch. Perhaps there is another bug, or something I simply misconfigured (I am rather new at this). I am running on a single MP machine; don't know if that makes any difference.
As I said, I'm agnostic about the solution, but given discussions about this on the Nutch forum, there seem to be others who see the same problem.
> Tasks launched by tasktracker in separate JVM can't generate log output
> -----------------------------------------------------------------------
>
> Key: HADOOP-406
> URL: http://issues.apache.org/jira/browse/HADOOP-406
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.4.0
> Reporter: Chris Schneider
> Attachments: HADOOP-406.patch
>
>
> Child JVM's don't have access to logging config system properties. When the child JVM gets launched, it doesn't inherit the Java system properties hadoop.log.dir and hadoop.log.file (which are actually based on the Bash environment variables $HADOOP_LOG_DIR and $HADOOP_LOGFILE). This means that you get no log messages from the actual map/reduce tasks that are executing.
> Stefan Groschupf reported this problem a while back:
> -------------------------------------------------------------------------
> To: hadoop-dev@lucene.apache.org
> From: Stefan Groschupf <sg...@media-style.com>
> Subject: tasks can't log bug?
> Date: Tue, 25 Jul 2006 19:26:17 -0700
> X-Virus-Checked: Checked by ClamAV on apache.org
> Hi Hadoop developers,
> I'm confused about the way logging works within map or reduce tasks.
> Since tasks are launched in a new JVM the java system properties "hadoop.log.dir" and "hadoop.log.file" are not passed to the new JVM.
> This prevents the child process from logging properly. In fact you get:
> java.io.FileNotFoundException: / (Is a directory)
> at java.io.FileOutputStream.openAppend(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:177)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:102)
> at org.apache.log4j.FileAppender.setFile(FileAppender.java:289)
> at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:165)
> at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:163)
> at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:256)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:132)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:96)
> at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:654)
> at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:612)
> at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.j
> 2006-07-25 15:59:07,553 INFO mapred.TaskTracker (TaskTracker.java:main(993)) - Child
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:415)
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:441)
> at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:4
> at org.apache.log4j.LogManager.<clinit>(LogManager.java:122)
> at org.apache.log4j.Logger.getLogger(Logger.java:104)
> at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229)
> at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImp
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAcc
> at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
> at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529
> at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235
> at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:370)
> at org.apache.hadoop.mapred.TaskTracker.<clinit>(TaskTracker.java:44)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:993)
> We see several ways to solve this problem. First retrieve the properties "hadoop.log.dir" and "hadoop.log.file" from the mother JVM and then pass them to the child JVM as within the args parameter.
> Second would be to access the environment variables "$HADOOP_LOG_DIR" and "$HADOOP_LOGFILE" using System.getEnv (java 1.5).
> Third there would be a more general solution. Taskrunner would resolve any environment variables it found in "mapred.child.java.opts" by lookup the value using System.getEnv().
> Eg:
> unix:
> export MAX_MEMORY = 200
> hadoop-site.xml:
> <name>mapred.child.java.opts</name>
> <value>-Xmx${MAX_MEMORY}</value>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-406) Tasks launched by tasktracker in
separate JVM can't generate log output
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-406?page=comments#action_12431083 ]
Doug Cutting commented on HADOOP-406:
-------------------------------------
Since I don't see this, it is hard for me to guess what's going on. Perhaps you can tell more about your installation: OS, JVM, config, etc. Thanks.
> Tasks launched by tasktracker in separate JVM can't generate log output
> -----------------------------------------------------------------------
>
> Key: HADOOP-406
> URL: http://issues.apache.org/jira/browse/HADOOP-406
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.4.0
> Reporter: Chris Schneider
> Attachments: HADOOP-406.patch
>
>
> Child JVM's don't have access to logging config system properties. When the child JVM gets launched, it doesn't inherit the Java system properties hadoop.log.dir and hadoop.log.file (which are actually based on the Bash environment variables $HADOOP_LOG_DIR and $HADOOP_LOGFILE). This means that you get no log messages from the actual map/reduce tasks that are executing.
> Stefan Groschupf reported this problem a while back:
> -------------------------------------------------------------------------
> To: hadoop-dev@lucene.apache.org
> From: Stefan Groschupf <sg...@media-style.com>
> Subject: tasks can't log bug?
> Date: Tue, 25 Jul 2006 19:26:17 -0700
> X-Virus-Checked: Checked by ClamAV on apache.org
> Hi Hadoop developers,
> I'm confused about the way logging works within map or reduce tasks.
> Since tasks are launched in a new JVM the java system properties "hadoop.log.dir" and "hadoop.log.file" are not passed to the new JVM.
> This prevents the child process from logging properly. In fact you get:
> java.io.FileNotFoundException: / (Is a directory)
> at java.io.FileOutputStream.openAppend(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:177)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:102)
> at org.apache.log4j.FileAppender.setFile(FileAppender.java:289)
> at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:165)
> at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:163)
> at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:256)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:132)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:96)
> at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:654)
> at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:612)
> at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.j
> 2006-07-25 15:59:07,553 INFO mapred.TaskTracker (TaskTracker.java:main(993)) - Child
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:415)
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:441)
> at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:4
> at org.apache.log4j.LogManager.<clinit>(LogManager.java:122)
> at org.apache.log4j.Logger.getLogger(Logger.java:104)
> at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229)
> at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImp
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAcc
> at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
> at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529
> at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235
> at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:370)
> at org.apache.hadoop.mapred.TaskTracker.<clinit>(TaskTracker.java:44)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:993)
> We see several ways to solve this problem. First retrieve the properties "hadoop.log.dir" and "hadoop.log.file" from the mother JVM and then pass them to the child JVM as within the args parameter.
> Second would be to access the environment variables "$HADOOP_LOG_DIR" and "$HADOOP_LOGFILE" using System.getEnv (java 1.5).
> Third there would be a more general solution. Taskrunner would resolve any environment variables it found in "mapred.child.java.opts" by lookup the value using System.getEnv().
> Eg:
> unix:
> export MAX_MEMORY = 200
> hadoop-site.xml:
> <name>mapred.child.java.opts</name>
> <value>-Xmx${MAX_MEMORY}</value>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-406) Tasks launched by tasktracker in
separate JVM can't generate log output
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-406?page=comments#action_12431047 ]
Doug Cutting commented on HADOOP-406:
-------------------------------------
The way this is currently supposed to work is that child processes log to standard output, and standard output is logged by the parent process (the task tracker). Is that not working for you?
Longer-term, we probably do not want child processes opening log files directly. Rather, they should be configured with a logger that, for each line logged, makes an RPC to the parent process, passing the level and log message. Then the tasktracker can log things accordingly.
> Tasks launched by tasktracker in separate JVM can't generate log output
> -----------------------------------------------------------------------
>
> Key: HADOOP-406
> URL: http://issues.apache.org/jira/browse/HADOOP-406
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.4.0
> Reporter: Chris Schneider
> Attachments: HADOOP-406.patch
>
>
> Child JVM's don't have access to logging config system properties. When the child JVM gets launched, it doesn't inherit the Java system properties hadoop.log.dir and hadoop.log.file (which are actually based on the Bash environment variables $HADOOP_LOG_DIR and $HADOOP_LOGFILE). This means that you get no log messages from the actual map/reduce tasks that are executing.
> Stefan Groschupf reported this problem a while back:
> -------------------------------------------------------------------------
> To: hadoop-dev@lucene.apache.org
> From: Stefan Groschupf <sg...@media-style.com>
> Subject: tasks can't log bug?
> Date: Tue, 25 Jul 2006 19:26:17 -0700
> X-Virus-Checked: Checked by ClamAV on apache.org
> Hi Hadoop developers,
> I'm confused about the way logging works within map or reduce tasks.
> Since tasks are launched in a new JVM the java system properties "hadoop.log.dir" and "hadoop.log.file" are not passed to the new JVM.
> This prevents the child process from logging properly. In fact you get:
> java.io.FileNotFoundException: / (Is a directory)
> at java.io.FileOutputStream.openAppend(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:177)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:102)
> at org.apache.log4j.FileAppender.setFile(FileAppender.java:289)
> at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:165)
> at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:163)
> at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:256)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:132)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:96)
> at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:654)
> at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:612)
> at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.j
> 2006-07-25 15:59:07,553 INFO mapred.TaskTracker (TaskTracker.java:main(993)) - Child
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:415)
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:441)
> at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:4
> at org.apache.log4j.LogManager.<clinit>(LogManager.java:122)
> at org.apache.log4j.Logger.getLogger(Logger.java:104)
> at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229)
> at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImp
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAcc
> at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
> at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529
> at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235
> at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:370)
> at org.apache.hadoop.mapred.TaskTracker.<clinit>(TaskTracker.java:44)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:993)
> We see several ways to solve this problem. First retrieve the properties "hadoop.log.dir" and "hadoop.log.file" from the mother JVM and then pass them to the child JVM as within the args parameter.
> Second would be to access the environment variables "$HADOOP_LOG_DIR" and "$HADOOP_LOGFILE" using System.getEnv (java 1.5).
> Third there would be a more general solution. Taskrunner would resolve any environment variables it found in "mapred.child.java.opts" by lookup the value using System.getEnv().
> Eg:
> unix:
> export MAX_MEMORY = 200
> hadoop-site.xml:
> <name>mapred.child.java.opts</name>
> <value>-Xmx${MAX_MEMORY}</value>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-406) Tasks launched by tasktracker in
separate JVM can't generate log output
Posted by "Dominik Friedrich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12461953 ]
Dominik Friedrich commented on HADOOP-406:
------------------------------------------
Please ignore my previous comment. It worked after the upgrade only because there were no mapred errors that had to be logged. I should have read the all comments to figure out that the log appender causes this problem.
> Tasks launched by tasktracker in separate JVM can't generate log output
> -----------------------------------------------------------------------
>
> Key: HADOOP-406
> URL: https://issues.apache.org/jira/browse/HADOOP-406
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.4.0
> Reporter: Chris Schneider
> Assigned To: Owen O'Malley
> Attachments: HADOOP-406.patch
>
>
> Child JVM's don't have access to logging config system properties. When the child JVM gets launched, it doesn't inherit the Java system properties hadoop.log.dir and hadoop.log.file (which are actually based on the Bash environment variables $HADOOP_LOG_DIR and $HADOOP_LOGFILE). This means that you get no log messages from the actual map/reduce tasks that are executing.
> Stefan Groschupf reported this problem a while back:
> -------------------------------------------------------------------------
> To: hadoop-dev@lucene.apache.org
> From: Stefan Groschupf <sg...@media-style.com>
> Subject: tasks can't log bug?
> Date: Tue, 25 Jul 2006 19:26:17 -0700
> X-Virus-Checked: Checked by ClamAV on apache.org
> Hi Hadoop developers,
> I'm confused about the way logging works within map or reduce tasks.
> Since tasks are launched in a new JVM the java system properties "hadoop.log.dir" and "hadoop.log.file" are not passed to the new JVM.
> This prevents the child process from logging properly. In fact you get:
> java.io.FileNotFoundException: / (Is a directory)
> at java.io.FileOutputStream.openAppend(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:177)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:102)
> at org.apache.log4j.FileAppender.setFile(FileAppender.java:289)
> at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:165)
> at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:163)
> at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:256)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:132)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:96)
> at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:654)
> at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:612)
> at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.j
> 2006-07-25 15:59:07,553 INFO mapred.TaskTracker (TaskTracker.java:main(993)) - Child
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:415)
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:441)
> at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:4
> at org.apache.log4j.LogManager.<clinit>(LogManager.java:122)
> at org.apache.log4j.Logger.getLogger(Logger.java:104)
> at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229)
> at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImp
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAcc
> at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
> at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529
> at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235
> at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:370)
> at org.apache.hadoop.mapred.TaskTracker.<clinit>(TaskTracker.java:44)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:993)
> We see several ways to solve this problem. First retrieve the properties "hadoop.log.dir" and "hadoop.log.file" from the mother JVM and then pass them to the child JVM as within the args parameter.
> Second would be to access the environment variables "$HADOOP_LOG_DIR" and "$HADOOP_LOGFILE" using System.getEnv (java 1.5).
> Third there would be a more general solution. Taskrunner would resolve any environment variables it found in "mapred.child.java.opts" by lookup the value using System.getEnv().
> Eg:
> unix:
> export MAX_MEMORY = 200
> hadoop-site.xml:
> <name>mapred.child.java.opts</name>
> <value>-Xmx${MAX_MEMORY}</value>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-406) Tasks launched by tasktracker in
separate JVM can't generate log output
Posted by "Chris Schneider (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-406?page=all ]
Chris Schneider updated HADOOP-406:
-----------------------------------
Attachment: HADOOP-406.patch
Here's a patch that provides one solution to the problem.
I modified TaskRunner.java so that it supports elements like -Dhadoop.log.dir=@property[hadoop.log.dir]@ within the mapred.child.java.opts Hadoop configuration property (replacing @property[hadoop.log.dir]@ with this property value).
If others agree that this is the appropriate way to solve the problem, then we should probably modify mapred.child.java.opts in hadoop-default.xml to include all 5 of the items in the default $HADOOP_OPTS (hadoop.log.dir, hadoop.log.file, hadoop.home.dir, hadoop.id.str, and hadoop.root.logger), using this method. A few notes, though:
1) This creates yet another dependency on the contents of $HADOOP_OPTS (i.e., anyone that adds items to this should probably add them to mapred.child.java.opts as well.) This is less than elegant.
2) It doesn't seem like hadoop.home.dir, hadoop.id.str, and hadoop.root.logger are actually being used currently by the code. Both hadoop.home.dir and hadoop.id.str are loaded by LogFormatter.initFileHandler(), but that method doesn't seem to be used by anyone. It doesn't seem like anyone is loading hadoop.root.logger at all.
> Tasks launched by tasktracker in separate JVM can't generate log output
> -----------------------------------------------------------------------
>
> Key: HADOOP-406
> URL: http://issues.apache.org/jira/browse/HADOOP-406
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.4.0
> Reporter: Chris Schneider
> Attachments: HADOOP-406.patch
>
>
> Child JVM's don't have access to logging config system properties. When the child JVM gets launched, it doesn't inherit the Java system properties hadoop.log.dir and hadoop.log.file (which are actually based on the Bash environment variables $HADOOP_LOG_DIR and $HADOOP_LOGFILE). This means that you get no log messages from the actual map/reduce tasks that are executing.
> Stefan Groschupf reported this problem a while back:
> -------------------------------------------------------------------------
> To: hadoop-dev@lucene.apache.org
> From: Stefan Groschupf <sg...@media-style.com>
> Subject: tasks can't log bug?
> Date: Tue, 25 Jul 2006 19:26:17 -0700
> X-Virus-Checked: Checked by ClamAV on apache.org
> Hi Hadoop developers,
> I'm confused about the way logging works within map or reduce tasks.
> Since tasks are launched in a new JVM the java system properties "hadoop.log.dir" and "hadoop.log.file" are not passed to the new JVM.
> This prevents the child process from logging properly. In fact you get:
> java.io.FileNotFoundException: / (Is a directory)
> at java.io.FileOutputStream.openAppend(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:177)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:102)
> at org.apache.log4j.FileAppender.setFile(FileAppender.java:289)
> at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:165)
> at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:163)
> at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:256)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:132)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:96)
> at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:654)
> at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:612)
> at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.j
> 2006-07-25 15:59:07,553 INFO mapred.TaskTracker (TaskTracker.java:main(993)) - Child
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:415)
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:441)
> at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:4
> at org.apache.log4j.LogManager.<clinit>(LogManager.java:122)
> at org.apache.log4j.Logger.getLogger(Logger.java:104)
> at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229)
> at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImp
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAcc
> at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
> at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529
> at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235
> at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:370)
> at org.apache.hadoop.mapred.TaskTracker.<clinit>(TaskTracker.java:44)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:993)
> We see several ways to solve this problem. First retrieve the properties "hadoop.log.dir" and "hadoop.log.file" from the mother JVM and then pass them to the child JVM as within the args parameter.
> Second would be to access the environment variables "$HADOOP_LOG_DIR" and "$HADOOP_LOGFILE" using System.getEnv (java 1.5).
> Third there would be a more general solution. Taskrunner would resolve any environment variables it found in "mapred.child.java.opts" by lookup the value using System.getEnv().
> Eg:
> unix:
> export MAX_MEMORY = 200
> hadoop-site.xml:
> <name>mapred.child.java.opts</name>
> <value>-Xmx${MAX_MEMORY}</value>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HADOOP-406) Tasks launched by tasktracker in
separate JVM can't generate log output
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley resolved HADOOP-406.
----------------------------------
Resolution: Duplicate
Assignee: (was: Owen O'Malley)
This was resolved by giving the child jvm a special logger that captures it as part of the job and provides it via http.
> Tasks launched by tasktracker in separate JVM can't generate log output
> -----------------------------------------------------------------------
>
> Key: HADOOP-406
> URL: https://issues.apache.org/jira/browse/HADOOP-406
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.4.0
> Reporter: Chris Schneider
> Attachments: HADOOP-406.patch
>
>
> Child JVM's don't have access to logging config system properties. When the child JVM gets launched, it doesn't inherit the Java system properties hadoop.log.dir and hadoop.log.file (which are actually based on the Bash environment variables $HADOOP_LOG_DIR and $HADOOP_LOGFILE). This means that you get no log messages from the actual map/reduce tasks that are executing.
> Stefan Groschupf reported this problem a while back:
> -------------------------------------------------------------------------
> To: hadoop-dev@lucene.apache.org
> From: Stefan Groschupf <sg...@media-style.com>
> Subject: tasks can't log bug?
> Date: Tue, 25 Jul 2006 19:26:17 -0700
> X-Virus-Checked: Checked by ClamAV on apache.org
> Hi Hadoop developers,
> I'm confused about the way logging works within map or reduce tasks.
> Since tasks are launched in a new JVM the java system properties "hadoop.log.dir" and "hadoop.log.file" are not passed to the new JVM.
> This prevents the child process from logging properly. In fact you get:
> java.io.FileNotFoundException: / (Is a directory)
> at java.io.FileOutputStream.openAppend(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:177)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:102)
> at org.apache.log4j.FileAppender.setFile(FileAppender.java:289)
> at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:165)
> at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:163)
> at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:256)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:132)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:96)
> at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:654)
> at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:612)
> at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.j
> 2006-07-25 15:59:07,553 INFO mapred.TaskTracker (TaskTracker.java:main(993)) - Child
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:415)
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:441)
> at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:4
> at org.apache.log4j.LogManager.<clinit>(LogManager.java:122)
> at org.apache.log4j.Logger.getLogger(Logger.java:104)
> at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229)
> at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImp
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAcc
> at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
> at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529
> at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235
> at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:370)
> at org.apache.hadoop.mapred.TaskTracker.<clinit>(TaskTracker.java:44)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:993)
> We see several ways to solve this problem. First retrieve the properties "hadoop.log.dir" and "hadoop.log.file" from the mother JVM and then pass them to the child JVM as within the args parameter.
> Second would be to access the environment variables "$HADOOP_LOG_DIR" and "$HADOOP_LOGFILE" using System.getEnv (java 1.5).
> Third there would be a more general solution. Taskrunner would resolve any environment variables it found in "mapred.child.java.opts" by lookup the value using System.getEnv().
> Eg:
> unix:
> export MAX_MEMORY = 200
> hadoop-site.xml:
> <name>mapred.child.java.opts</name>
> <value>-Xmx${MAX_MEMORY}</value>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-406) Tasks launched by tasktracker in
separate JVM can't generate log output
Posted by "Dominik Friedrich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12461934 ]
Dominik Friedrich commented on HADOOP-406:
------------------------------------------
I ran into this problem with hadoop 0.9.2, too. Upgrading log4j to version 1.2.14 seems to have solved the problem.
rlog4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: / (Invalid argument)
at java.io.FileOutputStream.openAppend(Native Method)
at java.io.FileOutputStream.(FileOutputStream.java:177)
at java.io.FileOutputStream.(FileOutputStream.java:102)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:289)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:163)
at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:215)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:256)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:132)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:96)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:654)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:612)
at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:509)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:415)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:441)
at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:468)
at org.apache.log4j.LogManager.(LogManager.java:122)
at org.apache.log4j.Logger.getLogger(Logger.java:104)
at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229)
at org.apache.commons.logging.impl.Log4JLogger.(Log4JLogger.java:65)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529)
at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235)
at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:370)
at org.apache.hadoop.mapred.TaskTracker.(TaskTracker.java:57)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1367)
log4j:ERROR Either File or DatePattern options are not set for appender [DRFA].
> Tasks launched by tasktracker in separate JVM can't generate log output
> -----------------------------------------------------------------------
>
> Key: HADOOP-406
> URL: https://issues.apache.org/jira/browse/HADOOP-406
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.4.0
> Reporter: Chris Schneider
> Assigned To: Owen O'Malley
> Attachments: HADOOP-406.patch
>
>
> Child JVM's don't have access to logging config system properties. When the child JVM gets launched, it doesn't inherit the Java system properties hadoop.log.dir and hadoop.log.file (which are actually based on the Bash environment variables $HADOOP_LOG_DIR and $HADOOP_LOGFILE). This means that you get no log messages from the actual map/reduce tasks that are executing.
> Stefan Groschupf reported this problem a while back:
> -------------------------------------------------------------------------
> To: hadoop-dev@lucene.apache.org
> From: Stefan Groschupf <sg...@media-style.com>
> Subject: tasks can't log bug?
> Date: Tue, 25 Jul 2006 19:26:17 -0700
> X-Virus-Checked: Checked by ClamAV on apache.org
> Hi Hadoop developers,
> I'm confused about the way logging works within map or reduce tasks.
> Since tasks are launched in a new JVM the java system properties "hadoop.log.dir" and "hadoop.log.file" are not passed to the new JVM.
> This prevents the child process from logging properly. In fact you get:
> java.io.FileNotFoundException: / (Is a directory)
> at java.io.FileOutputStream.openAppend(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:177)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:102)
> at org.apache.log4j.FileAppender.setFile(FileAppender.java:289)
> at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:165)
> at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:163)
> at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:256)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:132)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:96)
> at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:654)
> at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:612)
> at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.j
> 2006-07-25 15:59:07,553 INFO mapred.TaskTracker (TaskTracker.java:main(993)) - Child
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:415)
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:441)
> at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:4
> at org.apache.log4j.LogManager.<clinit>(LogManager.java:122)
> at org.apache.log4j.Logger.getLogger(Logger.java:104)
> at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229)
> at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImp
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAcc
> at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
> at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529
> at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235
> at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:370)
> at org.apache.hadoop.mapred.TaskTracker.<clinit>(TaskTracker.java:44)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:993)
> We see several ways to solve this problem. First retrieve the properties "hadoop.log.dir" and "hadoop.log.file" from the mother JVM and then pass them to the child JVM as within the args parameter.
> Second would be to access the environment variables "$HADOOP_LOG_DIR" and "$HADOOP_LOGFILE" using System.getEnv (java 1.5).
> Third there would be a more general solution. Taskrunner would resolve any environment variables it found in "mapred.child.java.opts" by lookup the value using System.getEnv().
> Eg:
> unix:
> export MAX_MEMORY = 200
> hadoop-site.xml:
> <name>mapred.child.java.opts</name>
> <value>-Xmx${MAX_MEMORY}</value>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-406) Tasks launched by tasktracker in
separate JVM can't generate log output
Posted by "Chris Schneider (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-406?page=comments#action_12439994 ]
Chris Schneider commented on HADOOP-406:
----------------------------------------
Sorry it took me so long to get back to this. I have just integrated Hadoop 0.5 and was about to back out my changes and install a more standard configuration when I noticed what I believe is the problem. The log4j.properties file we use doesn't default to the console logger like the standard log4j.properties does.
Our log4j.properties file:
# RootLogger - DailyRollingFileAppender
log4j.rootLogger=INFO,DRFA
Vanilla log4j.properties file:
hadoop.root.logger=INFO,console
Thus, when the child JVM is launched, it's going to try to use our DRFA logger, not send its log output to standard output. Note that before Owen fixed HADOOP-279, running without the hadoop script broke logging (probably in a similar way), and the child JVM is launched without the hadoop script.
Owen should probably comment on this, but it seems that the "child processes log to standard output, and standard output is logged by the parent process" design requires that every log4j.properties file must have console in its hadoop.root.logger.
> Tasks launched by tasktracker in separate JVM can't generate log output
> -----------------------------------------------------------------------
>
> Key: HADOOP-406
> URL: http://issues.apache.org/jira/browse/HADOOP-406
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.4.0
> Reporter: Chris Schneider
> Attachments: HADOOP-406.patch
>
>
> Child JVM's don't have access to logging config system properties. When the child JVM gets launched, it doesn't inherit the Java system properties hadoop.log.dir and hadoop.log.file (which are actually based on the Bash environment variables $HADOOP_LOG_DIR and $HADOOP_LOGFILE). This means that you get no log messages from the actual map/reduce tasks that are executing.
> Stefan Groschupf reported this problem a while back:
> -------------------------------------------------------------------------
> To: hadoop-dev@lucene.apache.org
> From: Stefan Groschupf <sg...@media-style.com>
> Subject: tasks can't log bug?
> Date: Tue, 25 Jul 2006 19:26:17 -0700
> X-Virus-Checked: Checked by ClamAV on apache.org
> Hi Hadoop developers,
> I'm confused about the way logging works within map or reduce tasks.
> Since tasks are launched in a new JVM the java system properties "hadoop.log.dir" and "hadoop.log.file" are not passed to the new JVM.
> This prevents the child process from logging properly. In fact you get:
> java.io.FileNotFoundException: / (Is a directory)
> at java.io.FileOutputStream.openAppend(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:177)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:102)
> at org.apache.log4j.FileAppender.setFile(FileAppender.java:289)
> at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:165)
> at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:163)
> at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:256)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:132)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:96)
> at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:654)
> at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:612)
> at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.j
> 2006-07-25 15:59:07,553 INFO mapred.TaskTracker (TaskTracker.java:main(993)) - Child
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:415)
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:441)
> at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:4
> at org.apache.log4j.LogManager.<clinit>(LogManager.java:122)
> at org.apache.log4j.Logger.getLogger(Logger.java:104)
> at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229)
> at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImp
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAcc
> at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
> at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529
> at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235
> at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:370)
> at org.apache.hadoop.mapred.TaskTracker.<clinit>(TaskTracker.java:44)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:993)
> We see several ways to solve this problem. First retrieve the properties "hadoop.log.dir" and "hadoop.log.file" from the mother JVM and then pass them to the child JVM as within the args parameter.
> Second would be to access the environment variables "$HADOOP_LOG_DIR" and "$HADOOP_LOGFILE" using System.getEnv (java 1.5).
> Third there would be a more general solution. Taskrunner would resolve any environment variables it found in "mapred.child.java.opts" by lookup the value using System.getEnv().
> Eg:
> unix:
> export MAX_MEMORY = 200
> hadoop-site.xml:
> <name>mapred.child.java.opts</name>
> <value>-Xmx${MAX_MEMORY}</value>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-406) Tasks launched by tasktracker in
separate JVM can't generate log output
Posted by "Doug Cook (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-406?page=comments#action_12430233 ]
Doug Cook commented on HADOOP-406:
----------------------------------
This issue bit me as well, and it took me some time to unravel the mystery. It would be nice to fix it in the main distribution!
I'm agnostic about what the long-term "right" solution is, but Chris's patch certainly works for me. (Thanks, Chris!)
Doug
> Tasks launched by tasktracker in separate JVM can't generate log output
> -----------------------------------------------------------------------
>
> Key: HADOOP-406
> URL: http://issues.apache.org/jira/browse/HADOOP-406
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.4.0
> Reporter: Chris Schneider
> Attachments: HADOOP-406.patch
>
>
> Child JVM's don't have access to logging config system properties. When the child JVM gets launched, it doesn't inherit the Java system properties hadoop.log.dir and hadoop.log.file (which are actually based on the Bash environment variables $HADOOP_LOG_DIR and $HADOOP_LOGFILE). This means that you get no log messages from the actual map/reduce tasks that are executing.
> Stefan Groschupf reported this problem a while back:
> -------------------------------------------------------------------------
> To: hadoop-dev@lucene.apache.org
> From: Stefan Groschupf <sg...@media-style.com>
> Subject: tasks can't log bug?
> Date: Tue, 25 Jul 2006 19:26:17 -0700
> X-Virus-Checked: Checked by ClamAV on apache.org
> Hi Hadoop developers,
> I'm confused about the way logging works within map or reduce tasks.
> Since tasks are launched in a new JVM the java system properties "hadoop.log.dir" and "hadoop.log.file" are not passed to the new JVM.
> This prevents the child process from logging properly. In fact you get:
> java.io.FileNotFoundException: / (Is a directory)
> at java.io.FileOutputStream.openAppend(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:177)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:102)
> at org.apache.log4j.FileAppender.setFile(FileAppender.java:289)
> at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:165)
> at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:163)
> at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:256)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:132)
> at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:96)
> at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:654)
> at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:612)
> at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.j
> 2006-07-25 15:59:07,553 INFO mapred.TaskTracker (TaskTracker.java:main(993)) - Child
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:415)
> at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:441)
> at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:4
> at org.apache.log4j.LogManager.<clinit>(LogManager.java:122)
> at org.apache.log4j.Logger.getLogger(Logger.java:104)
> at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229)
> at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImp
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAcc
> at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
> at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529
> at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235
> at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:370)
> at org.apache.hadoop.mapred.TaskTracker.<clinit>(TaskTracker.java:44)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:993)
> We see several ways to solve this problem. First retrieve the properties "hadoop.log.dir" and "hadoop.log.file" from the mother JVM and then pass them to the child JVM as within the args parameter.
> Second would be to access the environment variables "$HADOOP_LOG_DIR" and "$HADOOP_LOGFILE" using System.getEnv (java 1.5).
> Third there would be a more general solution. Taskrunner would resolve any environment variables it found in "mapred.child.java.opts" by lookup the value using System.getEnv().
> Eg:
> unix:
> export MAX_MEMORY = 200
> hadoop-site.xml:
> <name>mapred.child.java.opts</name>
> <value>-Xmx${MAX_MEMORY}</value>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira