You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Mahesh Khandewal <ma...@gmail.com> on 2014/04/13 07:17:27 UTC
Changing default scheduler in hadoop
Sir i am using Hadoop 1.1.2
I don't know where is the code residing of default scheduler?
I want to change the default scheduler to fair how can i do this??
And if i want to get back to default scheduler how can i do this?
I am struggling since 4 months to get help on Apache Hadoop??
I am new to this mailing list
Fwd: Changing default scheduler in hadoop
Posted by Mahesh Khandewal <ma...@gmail.com>.
---------- Forwarded message ----------
From: Mahesh Khandewal <ma...@gmail.com>
Date: Mon, 14 Apr 2014 08:42:16 +0530
Subject: Re: Changing default scheduler in hadoop
To: user@hadoop.apache.org
Cc: Ekta Agrawal <ek...@gmail.com>,
"common-user@hadoop.apache.org" <co...@hadoop.apache.org>,
"hdfs-user@hadoop.apache.org" <hd...@hadoop.apache.org>
Hi i have patch file of Resource Aware Scheduler
MAPREDUCE-1380_1.1.patch.txt<https://docs.google.com/file/d/0B11rCKdcyN82d1FuZENwZm12MTg/edit?usp=drive_web>
Now the patch contains the java code also xml code also. how to compile
and create a jar file for this patch??
I want to change the default fifo scheduler and run this resource aware
scheduler.
On Sun, Apr 13, 2014 at 10:54 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal
> <ma...@gmail.com> wrote:
> > Sir i am using Hadoop 1.1.2
> > I don't know where is the code residing of default scheduler?
>
> Doing a simple 'find' in the source checkout for name pattern
> 'Scheduler' should reveal pretty relevant hits. We do name our Java
> classes seriously :)
>
>
> https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
>
> > I want to change the default scheduler to fair how can i do this??
>
> You can override the mapred-site.xml placed property
> 'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a
> supplied one, such as Fair
> [http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity
> [http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
> Schedulers).
>
> > And if i want to get back to default scheduler how can i do this?
>
> Remove the configuration override, and it will always go back to the
> default FIFO based scheduler, the same whose source has been linked
> above.
>
> > I am struggling since 4 months to get help on Apache Hadoop??
>
> Are you unsure about this?
>
> --
> Harsh J
>
Fwd: Changing default scheduler in hadoop
Posted by Mahesh Khandewal <ma...@gmail.com>.
---------- Forwarded message ----------
From: Mahesh Khandewal <ma...@gmail.com>
Date: Mon, 14 Apr 2014 08:42:16 +0530
Subject: Re: Changing default scheduler in hadoop
To: user@hadoop.apache.org
Cc: Ekta Agrawal <ek...@gmail.com>,
"common-user@hadoop.apache.org" <co...@hadoop.apache.org>,
"hdfs-user@hadoop.apache.org" <hd...@hadoop.apache.org>
Hi i have patch file of Resource Aware Scheduler
MAPREDUCE-1380_1.1.patch.txt<https://docs.google.com/file/d/0B11rCKdcyN82d1FuZENwZm12MTg/edit?usp=drive_web>
Now the patch contains the java code also xml code also. how to compile
and create a jar file for this patch??
I want to change the default fifo scheduler and run this resource aware
scheduler.
On Sun, Apr 13, 2014 at 10:54 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal
> <ma...@gmail.com> wrote:
> > Sir i am using Hadoop 1.1.2
> > I don't know where is the code residing of default scheduler?
>
> Doing a simple 'find' in the source checkout for name pattern
> 'Scheduler' should reveal pretty relevant hits. We do name our Java
> classes seriously :)
>
>
> https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
>
> > I want to change the default scheduler to fair how can i do this??
>
> You can override the mapred-site.xml placed property
> 'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a
> supplied one, such as Fair
> [http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity
> [http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
> Schedulers).
>
> > And if i want to get back to default scheduler how can i do this?
>
> Remove the configuration override, and it will always go back to the
> default FIFO based scheduler, the same whose source has been linked
> above.
>
> > I am struggling since 4 months to get help on Apache Hadoop??
>
> Are you unsure about this?
>
> --
> Harsh J
>
Fwd: Changing default scheduler in hadoop
Posted by Mahesh Khandewal <ma...@gmail.com>.
---------- Forwarded message ----------
From: Mahesh Khandewal <ma...@gmail.com>
Date: Mon, 14 Apr 2014 08:42:16 +0530
Subject: Re: Changing default scheduler in hadoop
To: user@hadoop.apache.org
Cc: Ekta Agrawal <ek...@gmail.com>,
"common-user@hadoop.apache.org" <co...@hadoop.apache.org>,
"hdfs-user@hadoop.apache.org" <hd...@hadoop.apache.org>
Hi i have patch file of Resource Aware Scheduler
MAPREDUCE-1380_1.1.patch.txt<https://docs.google.com/file/d/0B11rCKdcyN82d1FuZENwZm12MTg/edit?usp=drive_web>
Now the patch contains the java code also xml code also. how to compile
and create a jar file for this patch??
I want to change the default fifo scheduler and run this resource aware
scheduler.
On Sun, Apr 13, 2014 at 10:54 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal
> <ma...@gmail.com> wrote:
> > Sir i am using Hadoop 1.1.2
> > I don't know where is the code residing of default scheduler?
>
> Doing a simple 'find' in the source checkout for name pattern
> 'Scheduler' should reveal pretty relevant hits. We do name our Java
> classes seriously :)
>
>
> https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
>
> > I want to change the default scheduler to fair how can i do this??
>
> You can override the mapred-site.xml placed property
> 'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a
> supplied one, such as Fair
> [http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity
> [http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
> Schedulers).
>
> > And if i want to get back to default scheduler how can i do this?
>
> Remove the configuration override, and it will always go back to the
> default FIFO based scheduler, the same whose source has been linked
> above.
>
> > I am struggling since 4 months to get help on Apache Hadoop??
>
> Are you unsure about this?
>
> --
> Harsh J
>
Fwd: Changing default scheduler in hadoop
Posted by Mahesh Khandewal <ma...@gmail.com>.
---------- Forwarded message ----------
From: Mahesh Khandewal <ma...@gmail.com>
Date: Mon, 14 Apr 2014 08:42:16 +0530
Subject: Re: Changing default scheduler in hadoop
To: user@hadoop.apache.org
Cc: Ekta Agrawal <ek...@gmail.com>,
"common-user@hadoop.apache.org" <co...@hadoop.apache.org>,
"hdfs-user@hadoop.apache.org" <hd...@hadoop.apache.org>
Hi i have patch file of Resource Aware Scheduler
MAPREDUCE-1380_1.1.patch.txt<https://docs.google.com/file/d/0B11rCKdcyN82d1FuZENwZm12MTg/edit?usp=drive_web>
Now the patch contains the java code also xml code also. how to compile
and create a jar file for this patch??
I want to change the default fifo scheduler and run this resource aware
scheduler.
On Sun, Apr 13, 2014 at 10:54 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal
> <ma...@gmail.com> wrote:
> > Sir i am using Hadoop 1.1.2
> > I don't know where is the code residing of default scheduler?
>
> Doing a simple 'find' in the source checkout for name pattern
> 'Scheduler' should reveal pretty relevant hits. We do name our Java
> classes seriously :)
>
>
> https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
>
> > I want to change the default scheduler to fair how can i do this??
>
> You can override the mapred-site.xml placed property
> 'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a
> supplied one, such as Fair
> [http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity
> [http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
> Schedulers).
>
> > And if i want to get back to default scheduler how can i do this?
>
> Remove the configuration override, and it will always go back to the
> default FIFO based scheduler, the same whose source has been linked
> above.
>
> > I am struggling since 4 months to get help on Apache Hadoop??
>
> Are you unsure about this?
>
> --
> Harsh J
>
Re: Changing default scheduler in hadoop
Posted by Mahesh Khandewal <ma...@gmail.com>.
Hi i have patch file of Resource Aware Scheduler
MAPREDUCE-1380_1.1.patch.txt<https://docs.google.com/file/d/0B11rCKdcyN82d1FuZENwZm12MTg/edit?usp=drive_web>
Now the patch contains the java code also xml code also. how to compile
and create a jar file for this patch??
I want to change the default fifo scheduler and run this resource aware
scheduler.
On Sun, Apr 13, 2014 at 10:54 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal
> <ma...@gmail.com> wrote:
> > Sir i am using Hadoop 1.1.2
> > I don't know where is the code residing of default scheduler?
>
> Doing a simple 'find' in the source checkout for name pattern
> 'Scheduler' should reveal pretty relevant hits. We do name our Java
> classes seriously :)
>
>
> https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
>
> > I want to change the default scheduler to fair how can i do this??
>
> You can override the mapred-site.xml placed property
> 'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a
> supplied one, such as Fair
> [http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity
> [http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
> Schedulers).
>
> > And if i want to get back to default scheduler how can i do this?
>
> Remove the configuration override, and it will always go back to the
> default FIFO based scheduler, the same whose source has been linked
> above.
>
> > I am struggling since 4 months to get help on Apache Hadoop??
>
> Are you unsure about this?
>
> --
> Harsh J
>
RE: Changing default scheduler in hadoop
Posted by dh...@wipro.com.
Hi Hadoop Users,
I am wondering if it is possible to use any other schedulers like SLURM/PBS Pro as default schedulers to scheduler MR Jobs
Regards
Dhvanika
-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Sunday, April 13, 2014 10:54 PM
To: <us...@hadoop.apache.org>
Cc: ektacloudstack@gmail.com; common-user@hadoop.apache.org; hdfs-user@hadoop.apache.org
Subject: Re: Changing default scheduler in hadoop
Hi,
On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal <ma...@gmail.com> wrote:
> Sir i am using Hadoop 1.1.2
> I don't know where is the code residing of default scheduler?
Doing a simple 'find' in the source checkout for name pattern 'Scheduler' should reveal pretty relevant hits. We do name our Java classes seriously :)
https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
> I want to change the default scheduler to fair how can i do this??
You can override the mapred-site.xml placed property 'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a supplied one, such as Fair [http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity [http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
Schedulers).
> And if i want to get back to default scheduler how can i do this?
Remove the configuration override, and it will always go back to the default FIFO based scheduler, the same whose source has been linked above.
> I am struggling since 4 months to get help on Apache Hadoop??
Are you unsure about this?
--
Harsh J
The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments.
WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.
www.wipro.com
Re: Changing default scheduler in hadoop
Posted by Mahesh Khandewal <ma...@gmail.com>.
Hi i have patch file of Resource Aware Scheduler
MAPREDUCE-1380_1.1.patch.txt<https://docs.google.com/file/d/0B11rCKdcyN82d1FuZENwZm12MTg/edit?usp=drive_web>
Now the patch contains the java code also xml code also. how to compile
and create a jar file for this patch??
I want to change the default fifo scheduler and run this resource aware
scheduler.
On Sun, Apr 13, 2014 at 10:54 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal
> <ma...@gmail.com> wrote:
> > Sir i am using Hadoop 1.1.2
> > I don't know where is the code residing of default scheduler?
>
> Doing a simple 'find' in the source checkout for name pattern
> 'Scheduler' should reveal pretty relevant hits. We do name our Java
> classes seriously :)
>
>
> https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
>
> > I want to change the default scheduler to fair how can i do this??
>
> You can override the mapred-site.xml placed property
> 'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a
> supplied one, such as Fair
> [http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity
> [http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
> Schedulers).
>
> > And if i want to get back to default scheduler how can i do this?
>
> Remove the configuration override, and it will always go back to the
> default FIFO based scheduler, the same whose source has been linked
> above.
>
> > I am struggling since 4 months to get help on Apache Hadoop??
>
> Are you unsure about this?
>
> --
> Harsh J
>
RE: Changing default scheduler in hadoop
Posted by dh...@wipro.com.
Hi Hadoop Users,
I am wondering if it is possible to use any other schedulers like SLURM/PBS Pro as default schedulers to scheduler MR Jobs
Regards
Dhvanika
-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Sunday, April 13, 2014 10:54 PM
To: <us...@hadoop.apache.org>
Cc: ektacloudstack@gmail.com; common-user@hadoop.apache.org; hdfs-user@hadoop.apache.org
Subject: Re: Changing default scheduler in hadoop
Hi,
On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal <ma...@gmail.com> wrote:
> Sir i am using Hadoop 1.1.2
> I don't know where is the code residing of default scheduler?
Doing a simple 'find' in the source checkout for name pattern 'Scheduler' should reveal pretty relevant hits. We do name our Java classes seriously :)
https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
> I want to change the default scheduler to fair how can i do this??
You can override the mapred-site.xml placed property 'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a supplied one, such as Fair [http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity [http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
Schedulers).
> And if i want to get back to default scheduler how can i do this?
Remove the configuration override, and it will always go back to the default FIFO based scheduler, the same whose source has been linked above.
> I am struggling since 4 months to get help on Apache Hadoop??
Are you unsure about this?
--
Harsh J
The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments.
WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.
www.wipro.com
RE: Changing default scheduler in hadoop
Posted by dh...@wipro.com.
Hi Hadoop Users,
I am wondering if it is possible to use any other schedulers like SLURM/PBS Pro as default schedulers to scheduler MR Jobs
Regards
Dhvanika
-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Sunday, April 13, 2014 10:54 PM
To: <us...@hadoop.apache.org>
Cc: ektacloudstack@gmail.com; common-user@hadoop.apache.org; hdfs-user@hadoop.apache.org
Subject: Re: Changing default scheduler in hadoop
Hi,
On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal <ma...@gmail.com> wrote:
> Sir i am using Hadoop 1.1.2
> I don't know where is the code residing of default scheduler?
Doing a simple 'find' in the source checkout for name pattern 'Scheduler' should reveal pretty relevant hits. We do name our Java classes seriously :)
https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
> I want to change the default scheduler to fair how can i do this??
You can override the mapred-site.xml placed property 'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a supplied one, such as Fair [http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity [http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
Schedulers).
> And if i want to get back to default scheduler how can i do this?
Remove the configuration override, and it will always go back to the default FIFO based scheduler, the same whose source has been linked above.
> I am struggling since 4 months to get help on Apache Hadoop??
Are you unsure about this?
--
Harsh J
The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments.
WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.
www.wipro.com
RE: Changing default scheduler in hadoop
Posted by dh...@wipro.com.
Hi Hadoop Users,
I am wondering if it is possible to use any other schedulers like SLURM/PBS Pro as default schedulers to scheduler MR Jobs
Regards
Dhvanika
-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Sunday, April 13, 2014 10:54 PM
To: <us...@hadoop.apache.org>
Cc: ektacloudstack@gmail.com; common-user@hadoop.apache.org; hdfs-user@hadoop.apache.org
Subject: Re: Changing default scheduler in hadoop
Hi,
On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal <ma...@gmail.com> wrote:
> Sir i am using Hadoop 1.1.2
> I don't know where is the code residing of default scheduler?
Doing a simple 'find' in the source checkout for name pattern 'Scheduler' should reveal pretty relevant hits. We do name our Java classes seriously :)
https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
> I want to change the default scheduler to fair how can i do this??
You can override the mapred-site.xml placed property 'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a supplied one, such as Fair [http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity [http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
Schedulers).
> And if i want to get back to default scheduler how can i do this?
Remove the configuration override, and it will always go back to the default FIFO based scheduler, the same whose source has been linked above.
> I am struggling since 4 months to get help on Apache Hadoop??
Are you unsure about this?
--
Harsh J
The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments.
WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.
www.wipro.com
Re: Changing default scheduler in hadoop
Posted by Mahesh Khandewal <ma...@gmail.com>.
Hi i have patch file of Resource Aware Scheduler
MAPREDUCE-1380_1.1.patch.txt<https://docs.google.com/file/d/0B11rCKdcyN82d1FuZENwZm12MTg/edit?usp=drive_web>
Now the patch contains the java code also xml code also. how to compile
and create a jar file for this patch??
I want to change the default fifo scheduler and run this resource aware
scheduler.
On Sun, Apr 13, 2014 at 10:54 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal
> <ma...@gmail.com> wrote:
> > Sir i am using Hadoop 1.1.2
> > I don't know where is the code residing of default scheduler?
>
> Doing a simple 'find' in the source checkout for name pattern
> 'Scheduler' should reveal pretty relevant hits. We do name our Java
> classes seriously :)
>
>
> https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
>
> > I want to change the default scheduler to fair how can i do this??
>
> You can override the mapred-site.xml placed property
> 'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a
> supplied one, such as Fair
> [http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity
> [http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
> Schedulers).
>
> > And if i want to get back to default scheduler how can i do this?
>
> Remove the configuration override, and it will always go back to the
> default FIFO based scheduler, the same whose source has been linked
> above.
>
> > I am struggling since 4 months to get help on Apache Hadoop??
>
> Are you unsure about this?
>
> --
> Harsh J
>
Re: Changing default scheduler in hadoop
Posted by Mahesh Khandewal <ma...@gmail.com>.
Hi i have patch file of Resource Aware Scheduler
MAPREDUCE-1380_1.1.patch.txt<https://docs.google.com/file/d/0B11rCKdcyN82d1FuZENwZm12MTg/edit?usp=drive_web>
Now the patch contains the java code also xml code also. how to compile
and create a jar file for this patch??
I want to change the default fifo scheduler and run this resource aware
scheduler.
On Sun, Apr 13, 2014 at 10:54 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal
> <ma...@gmail.com> wrote:
> > Sir i am using Hadoop 1.1.2
> > I don't know where is the code residing of default scheduler?
>
> Doing a simple 'find' in the source checkout for name pattern
> 'Scheduler' should reveal pretty relevant hits. We do name our Java
> classes seriously :)
>
>
> https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
>
> > I want to change the default scheduler to fair how can i do this??
>
> You can override the mapred-site.xml placed property
> 'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a
> supplied one, such as Fair
> [http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity
> [http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
> Schedulers).
>
> > And if i want to get back to default scheduler how can i do this?
>
> Remove the configuration override, and it will always go back to the
> default FIFO based scheduler, the same whose source has been linked
> above.
>
> > I am struggling since 4 months to get help on Apache Hadoop??
>
> Are you unsure about this?
>
> --
> Harsh J
>
Re: Changing default scheduler in hadoop
Posted by Harsh J <ha...@cloudera.com>.
Hi,
On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal
<ma...@gmail.com> wrote:
> Sir i am using Hadoop 1.1.2
> I don't know where is the code residing of default scheduler?
Doing a simple 'find' in the source checkout for name pattern
'Scheduler' should reveal pretty relevant hits. We do name our Java
classes seriously :)
https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
> I want to change the default scheduler to fair how can i do this??
You can override the mapred-site.xml placed property
'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a
supplied one, such as Fair
[http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity
[http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
Schedulers).
> And if i want to get back to default scheduler how can i do this?
Remove the configuration override, and it will always go back to the
default FIFO based scheduler, the same whose source has been linked
above.
> I am struggling since 4 months to get help on Apache Hadoop??
Are you unsure about this?
--
Harsh J
Re: Changing default scheduler in hadoop
Posted by Harsh J <ha...@cloudera.com>.
Hi,
On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal
<ma...@gmail.com> wrote:
> Sir i am using Hadoop 1.1.2
> I don't know where is the code residing of default scheduler?
Doing a simple 'find' in the source checkout for name pattern
'Scheduler' should reveal pretty relevant hits. We do name our Java
classes seriously :)
https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
> I want to change the default scheduler to fair how can i do this??
You can override the mapred-site.xml placed property
'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a
supplied one, such as Fair
[http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity
[http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
Schedulers).
> And if i want to get back to default scheduler how can i do this?
Remove the configuration override, and it will always go back to the
default FIFO based scheduler, the same whose source has been linked
above.
> I am struggling since 4 months to get help on Apache Hadoop??
Are you unsure about this?
--
Harsh J
Re: Changing default scheduler in hadoop
Posted by Harsh J <ha...@cloudera.com>.
Hi,
On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal
<ma...@gmail.com> wrote:
> Sir i am using Hadoop 1.1.2
> I don't know where is the code residing of default scheduler?
Doing a simple 'find' in the source checkout for name pattern
'Scheduler' should reveal pretty relevant hits. We do name our Java
classes seriously :)
https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
> I want to change the default scheduler to fair how can i do this??
You can override the mapred-site.xml placed property
'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a
supplied one, such as Fair
[http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity
[http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
Schedulers).
> And if i want to get back to default scheduler how can i do this?
Remove the configuration override, and it will always go back to the
default FIFO based scheduler, the same whose source has been linked
above.
> I am struggling since 4 months to get help on Apache Hadoop??
Are you unsure about this?
--
Harsh J
Re: Changing default scheduler in hadoop
Posted by Harsh J <ha...@cloudera.com>.
Hi,
On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal
<ma...@gmail.com> wrote:
> Sir i am using Hadoop 1.1.2
> I don't know where is the code residing of default scheduler?
Doing a simple 'find' in the source checkout for name pattern
'Scheduler' should reveal pretty relevant hits. We do name our Java
classes seriously :)
https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java
> I want to change the default scheduler to fair how can i do this??
You can override the mapred-site.xml placed property
'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a
supplied one, such as Fair
[http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity
[http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
Schedulers).
> And if i want to get back to default scheduler how can i do this?
Remove the configuration override, and it will always go back to the
default FIFO based scheduler, the same whose source has been linked
above.
> I am struggling since 4 months to get help on Apache Hadoop??
Are you unsure about this?
--
Harsh J