You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by yun peng <pe...@gmail.com> on 2013/06/23 13:39:09 UTC
Any mechanism in Hadoop to run in background
Hi, All...
We have a user case intended to run Mapreduce in background, while the
server serves online operations. The MapReduce job may have lower priority
comparing to the online jobs..
I know this is a different use case of Mapreduce comparing to its
originally targeted scenario (where Mapreduce largely own resource
exclusively)... But I want to know if there is any tuning knobs that allow
Mapreduce to run in low priority/with limited resource.
Thanks,
Yun
Re: Any mechanism in Hadoop to run in background
Posted by Harsh J <ha...@cloudera.com>.
You can configure a scheduler (Fair or Capacity) to have a pool/queue
with lower resources and submit your job to such a pool/queue for a
controlled execution.
On Sun, Jun 23, 2013 at 5:09 PM, yun peng <pe...@gmail.com> wrote:
> Hi, All...
> We have a user case intended to run Mapreduce in background, while the
> server serves online operations. The MapReduce job may have lower priority
> comparing to the online jobs..
>
> I know this is a different use case of Mapreduce comparing to its originally
> targeted scenario (where Mapreduce largely own resource exclusively)... But
> I want to know if there is any tuning knobs that allow Mapreduce to run in
> low priority/with limited resource.
>
> Thanks,
> Yun
--
Harsh J
Re: Any mechanism in Hadoop to run in background
Posted by Harsh J <ha...@cloudera.com>.
You can configure a scheduler (Fair or Capacity) to have a pool/queue
with lower resources and submit your job to such a pool/queue for a
controlled execution.
On Sun, Jun 23, 2013 at 5:09 PM, yun peng <pe...@gmail.com> wrote:
> Hi, All...
> We have a user case intended to run Mapreduce in background, while the
> server serves online operations. The MapReduce job may have lower priority
> comparing to the online jobs..
>
> I know this is a different use case of Mapreduce comparing to its originally
> targeted scenario (where Mapreduce largely own resource exclusively)... But
> I want to know if there is any tuning knobs that allow Mapreduce to run in
> low priority/with limited resource.
>
> Thanks,
> Yun
--
Harsh J
Re: Any mechanism in Hadoop to run in background
Posted by Harsh J <ha...@cloudera.com>.
You can configure a scheduler (Fair or Capacity) to have a pool/queue
with lower resources and submit your job to such a pool/queue for a
controlled execution.
On Sun, Jun 23, 2013 at 5:09 PM, yun peng <pe...@gmail.com> wrote:
> Hi, All...
> We have a user case intended to run Mapreduce in background, while the
> server serves online operations. The MapReduce job may have lower priority
> comparing to the online jobs..
>
> I know this is a different use case of Mapreduce comparing to its originally
> targeted scenario (where Mapreduce largely own resource exclusively)... But
> I want to know if there is any tuning knobs that allow Mapreduce to run in
> low priority/with limited resource.
>
> Thanks,
> Yun
--
Harsh J
Re: Any mechanism in Hadoop to run in background
Posted by Harsh J <ha...@cloudera.com>.
You can configure a scheduler (Fair or Capacity) to have a pool/queue
with lower resources and submit your job to such a pool/queue for a
controlled execution.
On Sun, Jun 23, 2013 at 5:09 PM, yun peng <pe...@gmail.com> wrote:
> Hi, All...
> We have a user case intended to run Mapreduce in background, while the
> server serves online operations. The MapReduce job may have lower priority
> comparing to the online jobs..
>
> I know this is a different use case of Mapreduce comparing to its originally
> targeted scenario (where Mapreduce largely own resource exclusively)... But
> I want to know if there is any tuning knobs that allow Mapreduce to run in
> low priority/with limited resource.
>
> Thanks,
> Yun
--
Harsh J