Phase level Scheduling algorithm in heterogeneous cluster

[Ahmed Altaj <ah...@yahoo.com.INVALID>]

Hi,
I'm trying to deploy PRISM algorithm which proposed in IEEE paper title "prism: Fine-Grained Resource-Aware Scheduling for MapReduce", for phase level scheduling work  in homogeneous cluster.I'm stack with first step how to implement this algorithm and configure it, which parts i have to change.
Best Regards...Ahmed 

Re: Phase level Scheduling algorithm in heterogeneous cluster

[Hariharan <ha...@gmail.com>]

Regarding the implementation - without knowing much about the algorithm - I
would guess you would at minimum need to implement a new scheduler - see
here
<https://github.com/apache/hadoop/tree/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler>
for some examples (fair/capacity/fifo). You may also be able to modify one
of the existing schedulers if the new algorithm isn't too different. You
would likely also need to change the MRAppMaster
<https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java>
and the Mapper
<https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Mapper.java>
and Reducer
<https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Reducer.java>
classes to work with this algorithm.


HTH,
Hariharan

On Sat, Apr 22, 2017 at 7:04 AM, Ahmed Altaj <ah...@yahoo.com.invalid>
wrote:

> Hi,
>
> I'm trying to deploy PRISM algorithm which proposed in IEEE paper title
> "prism: Fine-Grained Resource-Aware Scheduling for MapReduce", for phase
> level scheduling work  in homogeneous cluster.
> I'm stack with first step how to implement this algorithm and configure
> it, which parts i have to change.
>
> Best Regards...
> Ahmed
>