You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ravi Gummadi (JIRA)" <ji...@apache.org> on 2009/10/05 12:06:31 UTC
[jira] Created: (MAPREDUCE-1057) java tasks are not honouring the
value of mapred.userlog.limit.kb
java tasks are not honouring the value of mapred.userlog.limit.kb
-----------------------------------------------------------------
Key: MAPREDUCE-1057
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1057
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: tasktracker
Affects Versions: 0.20.1, 0.21.0, 0.22.0
Reporter: Ravi Gummadi
Assignee: Ravi Gummadi
Fix For: 0.21.0, 0.22.0, 0.20.1
Constructor of JvmEnv missed initializing logSize, which will be used as tailLength when java command for the task is built.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1057) java tasks are not honouring the
value of mapred.userlog.limit.kb
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763428#action_12763428 ]
Vinod K V commented on MAPREDUCE-1057:
--------------------------------------
Test-case? Agreed originally it wasn't there, but it's good time we add one :)
> java tasks are not honouring the value of mapred.userlog.limit.kb
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1057
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1057
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.20.1, 0.21.0, 0.22.0
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.20.2
>
> Attachments: MR-1057.patch
>
>
> Constructor of JvmEnv missed initializing logSize, which will be used as tailLength when java command for the task is built.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1057) java tasks are not honouring the
value of mapred.userlog.limit.kb
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12769123#action_12769123 ]
Vinod K V commented on MAPREDUCE-1057:
--------------------------------------
bq. The new issue will be addressed in MAPREDUCE-1100
Though one can argue both ways, MAPREDUCE-1100 is more of adding more features and may or may not make it to 0.21. On the other hand, this issue is a bug in the existing feature and was introduced by jvm-reuse in 0.19. So this issue has to be fixed separately from MAPREDUCE-1100.
> java tasks are not honouring the value of mapred.userlog.limit.kb
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1057
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1057
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.20.1, 0.21.0, 0.22.0
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.20.2
>
> Attachments: MR-1057.patch, MR-1057.v1.patch
>
>
> Constructor of JvmEnv missed initializing logSize, which will be used as tailLength when java command for the task is built.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1057) java tasks are not honouring the
value of mapred.userlog.limit.kb
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated MAPREDUCE-1057:
-------------------------------------
Fix Version/s: (was: 0.20.2)
> java tasks are not honouring the value of mapred.userlog.limit.kb
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1057
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1057
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.20.1, 0.21.0, 0.22.0
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Attachments: MR-1057.patch, MR-1057.v1.patch
>
>
> Constructor of JvmEnv missed initializing logSize, which will be used as tailLength when java command for the task is built.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1057) java tasks are not honouring the
value of mapred.userlog.limit.kb
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768655#action_12768655 ]
Ravi Gummadi commented on MAPREDUCE-1057:
-----------------------------------------
Sorry. The new issue will be addressed in MAPREDUCE-1100 (not MAPREDUCE-1025).
> java tasks are not honouring the value of mapred.userlog.limit.kb
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1057
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1057
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.20.1, 0.21.0, 0.22.0
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.20.2
>
> Attachments: MR-1057.patch, MR-1057.v1.patch
>
>
> Constructor of JvmEnv missed initializing logSize, which will be used as tailLength when java command for the task is built.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1057) java tasks are not honouring the
value of mapred.userlog.limit.kb
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated MAPREDUCE-1057:
------------------------------------
Attachment: MR-1057.v1.patch
Attaching patch with unit test for the fix added.
Manually tested that stderr and stdout of tasks are truncated(when mapreduce.task.userlog.limit.kb is set) properly with this fix.
Please review and provide your comments.
> java tasks are not honouring the value of mapred.userlog.limit.kb
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1057
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1057
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.20.1, 0.21.0, 0.22.0
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.20.2
>
> Attachments: MR-1057.patch, MR-1057.v1.patch
>
>
> Constructor of JvmEnv missed initializing logSize, which will be used as tailLength when java command for the task is built.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1057) java tasks are not honouring the
value of mapred.userlog.limit.kb
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766456#action_12766456 ]
Vinod K V commented on MAPREDUCE-1057:
--------------------------------------
We should have a test that actually runs a child JVM that tries to log beyond the limit and then we should verify that only the last N kb of log file is retained.
> java tasks are not honouring the value of mapred.userlog.limit.kb
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1057
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1057
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.20.1, 0.21.0, 0.22.0
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.20.2
>
> Attachments: MR-1057.patch, MR-1057.v1.patch
>
>
> Constructor of JvmEnv missed initializing logSize, which will be used as tailLength when java command for the task is built.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1057) java tasks are not honouring the
value of mapred.userlog.limit.kb
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768641#action_12768641 ]
Ravi Gummadi commented on MAPREDUCE-1057:
-----------------------------------------
I tried adding a test that runs child JVM that tries to log beyond the specified limit, but it is not working because the index file log.index is created before stdout and stderr are written to(by "tail -c <limit>" command). stdout and stderr are of correct size when I manually checked. But because the content of log.index file is wrong, logs are not useful. This new issue will be addressed in MAPREDUCE-1025.
> java tasks are not honouring the value of mapred.userlog.limit.kb
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1057
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1057
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.20.1, 0.21.0, 0.22.0
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.20.2
>
> Attachments: MR-1057.patch, MR-1057.v1.patch
>
>
> Constructor of JvmEnv missed initializing logSize, which will be used as tailLength when java command for the task is built.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1057) java tasks are not honouring the
value of mapred.userlog.limit.kb
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated MAPREDUCE-1057:
------------------------------------
Attachment: MR-1057.patch
Attaching patch for trunk(and for 0.21) with the fix.
> java tasks are not honouring the value of mapred.userlog.limit.kb
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1057
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1057
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.20.1, 0.21.0, 0.22.0
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.20.2
>
> Attachments: MR-1057.patch
>
>
> Constructor of JvmEnv missed initializing logSize, which will be used as tailLength when java command for the task is built.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1057) java tasks are not honouring the
value of mapred.userlog.limit.kb
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated MAPREDUCE-1057:
----------------------------------------
Fix Version/s: (was: 0.22.0)
(was: 0.20.1)
(was: 0.21.0)
0.20.2
> java tasks are not honouring the value of mapred.userlog.limit.kb
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1057
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1057
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.20.1, 0.21.0, 0.22.0
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.20.2
>
>
> Constructor of JvmEnv missed initializing logSize, which will be used as tailLength when java command for the task is built.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.