You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org> on 2010/09/27 07:05:33 UTC
[jira] Resolved: (MAPREDUCE-44) Hang JobTracker, running out of
memory
[ https://issues.apache.org/jira/browse/MAPREDUCE-44?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu resolved MAPREDUCE-44.
----------------------------------------------
Assignee: (was: Amareshwari Sriramadasu)
Resolution: Not A Problem
Closing old ticket.
> Hang JobTracker, running out of memory
> --------------------------------------
>
> Key: MAPREDUCE-44
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-44
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Koji Noguchi
>
> This may be expected.
> Hang JobTracker with 1G heapsize, top showed 99% cpu.
> Ran about 80 jobs. Each with 2500 mappers 200 reducers. They finish quite fast. 3-4 mins avg per job.
> (200k tasks)
> How much memory does JobTracker use for 'completed' (but not expired) jobs ?
> jmap -heap showed
> {noformat}
> ...
> PS Old Generation
> capacity = 932118528 (888.9375MB)
> used = 932118528 (888.9375MB)
> ...
> {noformat}
> jmap -histo showed
> {noformat}
> num #instances #bytes class name
> --------------------------------------
> 1: 3974182 355869992 [C
> 2: 5216606 125198544 java.lang.String
> 3: 2238560 107450880 java.util.TreeMap
> 4: 463206 101673488 [B
> 5: 1979995 63359840 java.util.TreeMap$Entry
> 6: 248400 35769600 org.apache.hadoop.mapred.TaskInProgress
> 7: 308803 30898112 [Ljava.lang.Object;
> 8: 978240 23477760 org.apache.hadoop.mapred.Counters$CounterRec
> 9: 249876 19990080 org.apache.hadoop.mapred.TaskStatus
> 10: 248836 19906880 java.net.URI
> 11: 230337 16584264 org.apache.hadoop.mapred.MapTask
> ...
> {noformat}
> Log showing many heartbeat discarded messages
> {noformat}
> 2007-10-30 22:55:46,912 WARN org.apache.hadoop.ipc.Server: IPC Server handler 6 on 58567, call heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@1afb9c9, false, true, 3942) from 99.99.99.99:9999 discarded for being too old (2578616)
> {noformat}
> Is the solution either to increase the jobtracker heapsize or set shorter 'mapred.userlog.retain.hours' ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.