You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Александр Малышевский <ma...@iqmen.ru> on 2012/04/13 13:28:48 UTC
MultiGlobal scheduling in the Fair Scheduler
Hello.
I'm using hadoop 0.20.203.0 and fair scheduler. And I have some difficulties
concerning the lack of multiple tasks per heartbeat in fair scheduler in
version 0.20.203.0.
I've found fixed feature about it -
https://issues.apache.org/jira/browse/MAPREDUCE-548. Fix version is
<https://issues.apache.org/jira/browse/MAPREDUCE/fixforversion/12314045>
0.21.0. Version 0.20.203.0 was released several months later than 0.21.0,
but as I can see this feature was not included in 0.20.203.0.
I don't want to upgrade my hadoop to another version cause it may make me
change lots of code. So the question is can I upgrade only fair scheduler
jar without upgrading other libraries? I just need multiple tasks per
heartbeat in fair scheduler. What's the easiest way to get it? I tried to
upgrade fair scheduler jar to 1.0.2 but it didn't worked with the following
exception:
2012-04-12 17:33:42,001 FATAL org.apache.hadoop.mapred.JobTracker:
java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/TaskType
at
org.apache.hadoop.mapred.FairScheduler.<clinit>(FairScheduler.java:64)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:833)
at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1573)
at
org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:183)
at
org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:175)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:3702)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.mapreduce.TaskType
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 10 more
2012-04-12 17:33:42,002 INFO org.apache.hadoop.mapred.JobTracker:
SHUTDOWN_MSG: