[jira] Commented: (HADOOP-5005) Create a mock MapReduce cluster simulator to test schedulers

["Hemanth Yamijala (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12662612#action_12662612 ] 

Hemanth Yamijala commented on HADOOP-5005:
------------------------------------------

Matei, we recently built a MiniMR based cluster for the capacity scheduler. Please look at HADOOP-4830. Can you see if that's along the lines you wanted ? If yes, maybe we can explore changing that to support schedulers generically.

> Create a mock MapReduce cluster simulator to test schedulers
> ------------------------------------------------------------
>
>                 Key: HADOOP-5005
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5005
>             Project: Hadoop Core
>          Issue Type: Test
>          Components: test
>            Reporter: Matei Zaharia
>
> Currently the Hadoop schedulers use a FakeTaskTrackerManager to run tests which is both messy and doesn't really simulate trackers going up and down, tasks finishing at different times, faiulres, etc. It would be nice to have a simulated MapReduce cluster where tasks really do take different amounts of (simulated) time, trackers may be slow, tasks can be made to fail, etc. The existing TaskTrackerManager interface given to the schedulers, plus perhaps a mockable clock (e.g. the FairScheduler.Clock class) should be enough to do all this. The end result will be easier-to-write and more complex scheduler tests.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.