You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by kumar r <ku...@gmail.com> on 2016/02/22 07:31:58 UTC
Hadoop-2.7.2 MR job stuck in accepted state
Hi,
I have configured hadoop-2.7.2 pseudo-node cluster in windows. When i
submit a MR job, it works fine. But if i submit multiple MR jobs then only
one job runs at a time.
First job is in RUNNING state and all other job in ACCEPTED state even yarn
has enough memory (3 GB free out of 6 GB).
But the same works fine in hadoop-2.5.2
*yarn-site.xml*
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>6144</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>256</value>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-mb</name>
<value>2250</value>
</property>
Any new property to be set for running multiple job at a time?
Stack Overflow Question :
http://stackoverflow.com/questions/35545986/hadoop-2-7-2-mr-job-in-accepted-state
Thanks,
Kumar
Re: Hadoop-2.7.2 MR job stuck in accepted state
Posted by Varun Saxena <vs...@gmail.com>.
Hi Kumar,
This has to do with yarn.scheduler.capacity.maximum-am-resource-percent
which by default is 0.1
This parameter is used to control the number of concurrently running apps.
Refer to
https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html#Queue_Properties
Regards,
Varun Saxena
On Mon, Feb 22, 2016 at 12:01 PM, kumar r <ku...@gmail.com> wrote:
> Hi,
>
> I have configured hadoop-2.7.2 pseudo-node cluster in windows. When i
> submit a MR job, it works fine. But if i submit multiple MR jobs then only
> one job runs at a time.
>
> First job is in RUNNING state and all other job in ACCEPTED state even
> yarn has enough memory (3 GB free out of 6 GB).
> But the same works fine in hadoop-2.5.2
>
>
> *yarn-site.xml*
>
> <property>
> <name>yarn.nodemanager.resource.memory-mb</name>
> <value>6144</value>
> </property>
> <property>
> <name>yarn.scheduler.minimum-allocation-mb</name>
> <value>256</value>
> </property>
> <property>
> <name>yarn.scheduler.maximum-allocation-mb</name>
> <value>2250</value>
> </property>
>
> Any new property to be set for running multiple job at a time?
>
> Stack Overflow Question : http://stackoverflow.com/questions/35545986/hadoop-2-7-2-mr-job-in-accepted-state
>
> Thanks,
>
> Kumar
>
>
Re: Hadoop-2.7.2 MR job stuck in accepted state
Posted by Varun Saxena <vs...@gmail.com>.
Hi Kumar,
This has to do with yarn.scheduler.capacity.maximum-am-resource-percent
which by default is 0.1
This parameter is used to control the number of concurrently running apps.
Refer to
https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html#Queue_Properties
Regards,
Varun Saxena
On Mon, Feb 22, 2016 at 12:01 PM, kumar r <ku...@gmail.com> wrote:
> Hi,
>
> I have configured hadoop-2.7.2 pseudo-node cluster in windows. When i
> submit a MR job, it works fine. But if i submit multiple MR jobs then only
> one job runs at a time.
>
> First job is in RUNNING state and all other job in ACCEPTED state even
> yarn has enough memory (3 GB free out of 6 GB).
> But the same works fine in hadoop-2.5.2
>
>
> *yarn-site.xml*
>
> <property>
> <name>yarn.nodemanager.resource.memory-mb</name>
> <value>6144</value>
> </property>
> <property>
> <name>yarn.scheduler.minimum-allocation-mb</name>
> <value>256</value>
> </property>
> <property>
> <name>yarn.scheduler.maximum-allocation-mb</name>
> <value>2250</value>
> </property>
>
> Any new property to be set for running multiple job at a time?
>
> Stack Overflow Question : http://stackoverflow.com/questions/35545986/hadoop-2-7-2-mr-job-in-accepted-state
>
> Thanks,
>
> Kumar
>
>
Re: Hadoop-2.7.2 MR job stuck in accepted state
Posted by Varun Saxena <vs...@gmail.com>.
Hi Kumar,
This has to do with yarn.scheduler.capacity.maximum-am-resource-percent
which by default is 0.1
This parameter is used to control the number of concurrently running apps.
Refer to
https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html#Queue_Properties
Regards,
Varun Saxena
On Mon, Feb 22, 2016 at 12:01 PM, kumar r <ku...@gmail.com> wrote:
> Hi,
>
> I have configured hadoop-2.7.2 pseudo-node cluster in windows. When i
> submit a MR job, it works fine. But if i submit multiple MR jobs then only
> one job runs at a time.
>
> First job is in RUNNING state and all other job in ACCEPTED state even
> yarn has enough memory (3 GB free out of 6 GB).
> But the same works fine in hadoop-2.5.2
>
>
> *yarn-site.xml*
>
> <property>
> <name>yarn.nodemanager.resource.memory-mb</name>
> <value>6144</value>
> </property>
> <property>
> <name>yarn.scheduler.minimum-allocation-mb</name>
> <value>256</value>
> </property>
> <property>
> <name>yarn.scheduler.maximum-allocation-mb</name>
> <value>2250</value>
> </property>
>
> Any new property to be set for running multiple job at a time?
>
> Stack Overflow Question : http://stackoverflow.com/questions/35545986/hadoop-2-7-2-mr-job-in-accepted-state
>
> Thanks,
>
> Kumar
>
>
Re: Hadoop-2.7.2 MR job stuck in accepted state
Posted by Varun Saxena <vs...@gmail.com>.
Hi Kumar,
This has to do with yarn.scheduler.capacity.maximum-am-resource-percent
which by default is 0.1
This parameter is used to control the number of concurrently running apps.
Refer to
https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html#Queue_Properties
Regards,
Varun Saxena
On Mon, Feb 22, 2016 at 12:01 PM, kumar r <ku...@gmail.com> wrote:
> Hi,
>
> I have configured hadoop-2.7.2 pseudo-node cluster in windows. When i
> submit a MR job, it works fine. But if i submit multiple MR jobs then only
> one job runs at a time.
>
> First job is in RUNNING state and all other job in ACCEPTED state even
> yarn has enough memory (3 GB free out of 6 GB).
> But the same works fine in hadoop-2.5.2
>
>
> *yarn-site.xml*
>
> <property>
> <name>yarn.nodemanager.resource.memory-mb</name>
> <value>6144</value>
> </property>
> <property>
> <name>yarn.scheduler.minimum-allocation-mb</name>
> <value>256</value>
> </property>
> <property>
> <name>yarn.scheduler.maximum-allocation-mb</name>
> <value>2250</value>
> </property>
>
> Any new property to be set for running multiple job at a time?
>
> Stack Overflow Question : http://stackoverflow.com/questions/35545986/hadoop-2-7-2-mr-job-in-accepted-state
>
> Thanks,
>
> Kumar
>
>