Guidance Related to Yarn Scheduler Development

[Abhishek 2 <ab...@iiitm.ac.in>]

Hi

I am a Ph.D. research scholar in IIITM Gwalior and a newbie in hadoop. My
research topic is to develop an algorithm for efficient yarn scheduling. I
am trying with a heuristic optimization algorithm but not sure how to start
writing code. I am not able to understand which module or repsitory should
be edited.

It will be a great help if I get any guidance or simialr code to study on
it.
-- 
Abhishek Gupta
PhD Scholar
IIITM Gwalior

Re: Guidance Related to Yarn Scheduler Development

[Hariharan <ha...@gmail.com>]

The scheduler code lives in the hadoop-yarn-project here
<https://github.com/apache/hadoop/tree/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler>.
You'll find a few implementations in that directory - "fair" for the
FairScheduler and "capacity" for the CapacityScheduler for example. You can
look at these to get some ideas on how the existing schedulers work. If
you're writing a new one, start by extending the AbstractYarnScheduler and
take it forward from there.

Thanks,
Hariharan

On Thu, Jul 1, 2021 at 8:07 PM Abhishek 2 <ab...@iiitm.ac.in> wrote:

> Hi
>
> I am a Ph.D. research scholar in IIITM Gwalior and a newbie in hadoop. My
> research topic is to develop an algorithm for efficient yarn scheduling. I
> am trying with a heuristic optimization algorithm but not sure how to start
> writing code. I am not able to understand which module or repsitory should
> be edited.
>
> It will be a great help if I get any guidance or simialr code to study on
> it.
> --
> Abhishek Gupta
> PhD Scholar
> IIITM Gwalior
>