You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Krishna Kishore Bonagiri <wr...@gmail.com> on 2013/06/21 11:15:27 UTC
Requesting containers on a specific host
Hi,
I am trying to get container on a specific host, using setHostName(0 call
on ResourceRequest, but could not get allocated anything forever, which
just works fine when I change the node name to "*". I am working on a
single node cluster, and I am giving the name of the single node I have in
my cluster.
Is there any specific format that I need to give for setHostName(), why
is it not working...
Thanks,
Kishore
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Hi Arun,
I just found that setHostName() doesn't work on 2.0.0-alpha and works on
2.0..4-alpha, I didn't check the intermediate versions. I have verified it
by starting the daemons of respective versions, and modifying the
ApplicationMaster.java in the distributed shell example, and running a date
command on it. Is this something already known? or have I been doing
something wrong?
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> Hi,
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster.
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...
>
> Thanks,
> Kishore
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Yes Arun, the one I am giving is same as the one I see in RM's log also.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> Hi,
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster.
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...
>
> Thanks,
> Kishore
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: Requesting containers on a specific host
Posted by "devaraj.k@huawei.com" <de...@huawei.com>.
Hi Kishore,
As per the exception given, Node Manager is getting excluded. It might be
happening that you have configured the Node Manager in excluded file using
this configuration in Resource Manager.
Could you check this configuration in RM, is it configured with any file
and that file contains the NM address?
<property>
<description>Path to file with nodes to exclude.</description>
<name>yarn.resourcemanager.nodes.exclude-path</name>
<value></value>
</property>
and also can you cross check this configuration, if you have updated this
then the modification is proper or not.
<property>
<description>Path to file with nodes to include.</description>
<name>yarn.resourcemanager.nodes.include-path</name>
<value></value>
</property>
Thanks&Regards
Devaraj k
--
View this message in context: http://lucene.472066.n3.nabble.com/Requesting-containers-on-a-specific-host-tp4072089p4075779.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
I could resolve this error with a simple host name change to a fully
qualified one including the domain name. But to go ahead running some of my
old example code, I am observing at least the following changes :
1) ClientRMProtocol and AMRMProtocol are removed.
2) ContainerManager is removed.
3) YarnRemoteException is removed.
4) ContainerRequest is removed.
It looks like it is now compulsory to modify my application accordingly,
where as on earlier versions, I could have run the same application without
any modifications.
Thanks,
Kishore
On Fri, Jul 5, 2013 at 7:37 PM, Krishna Kishore Bonagiri <
write2kishore@gmail.com> wrote:
> Hi Devaraj,
>
> Thanks for pointing me to this RC. I am trying this out, and getting
> this error for NM to get started. My RM is running fine, but NM is failing
> and saying that it is disallowed by RM and received a SHUTDOWN message.
> Please give me clue to resolve this.
>
> 2013-07-05 09:49:20,043 ERROR
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Unexpected
> error starting NodeStatusUpdater
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN
> signal from Resourcemanager ,Registration of NodeManager failed, Message
> from ResourceManager: Disallowed NodeManager from
> isredeng.swg.usma.ibm.com, Sending SHUTDOWN signal to the NodeManager.
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:290)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:156)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:401)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:447)
>
> Thanks,
> Kishore
>
>
> On Fri, Jul 5, 2013 at 8:52 AM, Devaraj k <de...@huawei.com> wrote:
>
>> Hi Kishore,****
>>
>> ** **
>>
>> hadoop-2.1.0 beta release is in voting process now.****
>>
>> ** **
>>
>> You can try out from hadoop-2.1.0 beta RC
>> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ or you could
>> check the same with trunk build.****
>>
>> ** **
>>
>> Thanks****
>>
>> Devaraj k****
>>
>> ** **
>>
>> *From:* Krishna Kishore Bonagiri [mailto:write2kishore@gmail.com]
>> *Sent:* 04 July 2013 21:33
>>
>> *To:* user@hadoop.apache.org
>> *Subject:* Re: Requesting containers on a specific host****
>>
>> ** **
>>
>> Thanks Arun, it seems to be available with 2.1.0-beta, when will that be
>> released? Or if I want it now, could I get from the trunk?****
>>
>> ** **
>>
>> -Kishore****
>>
>> ** **
>>
>> On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:****
>>
>> To guarantee nodes on a specific container you need to use the whitelist
>> feature we added recently:****
>>
>> https://issues.apache.org/jira/browse/YARN-398****
>>
>> Arun****
>>
>> ** **
>>
>> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
>> write2kishore@gmail.com> wrote:****
>>
>>
>>
>> ****
>>
>> I could get containers on specific nodes using addContainerRequest() on
>> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
>> in my cluster. And, my Application Master is trying to get 3 containers on
>> node1, and 3 containers on node2 in that order. ****
>>
>> ** **
>>
>> While trying to request on node1, it sometimes gives me those on node2,
>> and vice verse. When I get a container on a different node than the one I
>> need, I release it and make a fresh request. I am having to do like that
>> forever to get a container on the node I need. ****
>>
>> ** **
>>
>> Though the node I am requesting has enough resources, why does it keep
>> giving me containers on the other node? How can I make sure I get a
>> container on the node I want? ****
>>
>> ** **
>>
>> Note: I am using the default scheduler, i.e. Capacity Scheduler.****
>>
>> ** **
>>
>> Thanks,****
>>
>> Kishore****
>>
>> ** **
>>
>> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:****
>>
>> Check if the hostname you are setting is the same in the RM logs…****
>>
>> ** **
>>
>> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
>> write2kishore@gmail.com> wrote:****
>>
>>
>>
>> ****
>>
>> Hi,****
>>
>> I am trying to get container on a specific host, using setHostName(0
>> call on ResourceRequest, but could not get allocated anything forever,
>> which just works fine when I change the node name to "*". I am working on a
>> single node cluster, and I am giving the name of the single node I have in
>> my cluster. ****
>>
>> ** **
>>
>> Is there any specific format that I need to give for setHostName(), why
>> is it not working...****
>>
>> ** **
>>
>> Thanks,****
>>
>> Kishore****
>>
>> ** **
>>
>> --****
>>
>> Arun C. Murthy****
>>
>> Hortonworks Inc.
>> http://hortonworks.com/****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> --****
>>
>> Arun C. Murthy****
>>
>> Hortonworks Inc.
>> http://hortonworks.com/****
>>
>> ** **
>>
>> ** **
>>
>
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
I could resolve this error with a simple host name change to a fully
qualified one including the domain name. But to go ahead running some of my
old example code, I am observing at least the following changes :
1) ClientRMProtocol and AMRMProtocol are removed.
2) ContainerManager is removed.
3) YarnRemoteException is removed.
4) ContainerRequest is removed.
It looks like it is now compulsory to modify my application accordingly,
where as on earlier versions, I could have run the same application without
any modifications.
Thanks,
Kishore
On Fri, Jul 5, 2013 at 7:37 PM, Krishna Kishore Bonagiri <
write2kishore@gmail.com> wrote:
> Hi Devaraj,
>
> Thanks for pointing me to this RC. I am trying this out, and getting
> this error for NM to get started. My RM is running fine, but NM is failing
> and saying that it is disallowed by RM and received a SHUTDOWN message.
> Please give me clue to resolve this.
>
> 2013-07-05 09:49:20,043 ERROR
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Unexpected
> error starting NodeStatusUpdater
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN
> signal from Resourcemanager ,Registration of NodeManager failed, Message
> from ResourceManager: Disallowed NodeManager from
> isredeng.swg.usma.ibm.com, Sending SHUTDOWN signal to the NodeManager.
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:290)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:156)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:401)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:447)
>
> Thanks,
> Kishore
>
>
> On Fri, Jul 5, 2013 at 8:52 AM, Devaraj k <de...@huawei.com> wrote:
>
>> Hi Kishore,****
>>
>> ** **
>>
>> hadoop-2.1.0 beta release is in voting process now.****
>>
>> ** **
>>
>> You can try out from hadoop-2.1.0 beta RC
>> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ or you could
>> check the same with trunk build.****
>>
>> ** **
>>
>> Thanks****
>>
>> Devaraj k****
>>
>> ** **
>>
>> *From:* Krishna Kishore Bonagiri [mailto:write2kishore@gmail.com]
>> *Sent:* 04 July 2013 21:33
>>
>> *To:* user@hadoop.apache.org
>> *Subject:* Re: Requesting containers on a specific host****
>>
>> ** **
>>
>> Thanks Arun, it seems to be available with 2.1.0-beta, when will that be
>> released? Or if I want it now, could I get from the trunk?****
>>
>> ** **
>>
>> -Kishore****
>>
>> ** **
>>
>> On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:****
>>
>> To guarantee nodes on a specific container you need to use the whitelist
>> feature we added recently:****
>>
>> https://issues.apache.org/jira/browse/YARN-398****
>>
>> Arun****
>>
>> ** **
>>
>> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
>> write2kishore@gmail.com> wrote:****
>>
>>
>>
>> ****
>>
>> I could get containers on specific nodes using addContainerRequest() on
>> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
>> in my cluster. And, my Application Master is trying to get 3 containers on
>> node1, and 3 containers on node2 in that order. ****
>>
>> ** **
>>
>> While trying to request on node1, it sometimes gives me those on node2,
>> and vice verse. When I get a container on a different node than the one I
>> need, I release it and make a fresh request. I am having to do like that
>> forever to get a container on the node I need. ****
>>
>> ** **
>>
>> Though the node I am requesting has enough resources, why does it keep
>> giving me containers on the other node? How can I make sure I get a
>> container on the node I want? ****
>>
>> ** **
>>
>> Note: I am using the default scheduler, i.e. Capacity Scheduler.****
>>
>> ** **
>>
>> Thanks,****
>>
>> Kishore****
>>
>> ** **
>>
>> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:****
>>
>> Check if the hostname you are setting is the same in the RM logs…****
>>
>> ** **
>>
>> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
>> write2kishore@gmail.com> wrote:****
>>
>>
>>
>> ****
>>
>> Hi,****
>>
>> I am trying to get container on a specific host, using setHostName(0
>> call on ResourceRequest, but could not get allocated anything forever,
>> which just works fine when I change the node name to "*". I am working on a
>> single node cluster, and I am giving the name of the single node I have in
>> my cluster. ****
>>
>> ** **
>>
>> Is there any specific format that I need to give for setHostName(), why
>> is it not working...****
>>
>> ** **
>>
>> Thanks,****
>>
>> Kishore****
>>
>> ** **
>>
>> --****
>>
>> Arun C. Murthy****
>>
>> Hortonworks Inc.
>> http://hortonworks.com/****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> --****
>>
>> Arun C. Murthy****
>>
>> Hortonworks Inc.
>> http://hortonworks.com/****
>>
>> ** **
>>
>> ** **
>>
>
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
I could resolve this error with a simple host name change to a fully
qualified one including the domain name. But to go ahead running some of my
old example code, I am observing at least the following changes :
1) ClientRMProtocol and AMRMProtocol are removed.
2) ContainerManager is removed.
3) YarnRemoteException is removed.
4) ContainerRequest is removed.
It looks like it is now compulsory to modify my application accordingly,
where as on earlier versions, I could have run the same application without
any modifications.
Thanks,
Kishore
On Fri, Jul 5, 2013 at 7:37 PM, Krishna Kishore Bonagiri <
write2kishore@gmail.com> wrote:
> Hi Devaraj,
>
> Thanks for pointing me to this RC. I am trying this out, and getting
> this error for NM to get started. My RM is running fine, but NM is failing
> and saying that it is disallowed by RM and received a SHUTDOWN message.
> Please give me clue to resolve this.
>
> 2013-07-05 09:49:20,043 ERROR
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Unexpected
> error starting NodeStatusUpdater
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN
> signal from Resourcemanager ,Registration of NodeManager failed, Message
> from ResourceManager: Disallowed NodeManager from
> isredeng.swg.usma.ibm.com, Sending SHUTDOWN signal to the NodeManager.
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:290)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:156)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:401)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:447)
>
> Thanks,
> Kishore
>
>
> On Fri, Jul 5, 2013 at 8:52 AM, Devaraj k <de...@huawei.com> wrote:
>
>> Hi Kishore,****
>>
>> ** **
>>
>> hadoop-2.1.0 beta release is in voting process now.****
>>
>> ** **
>>
>> You can try out from hadoop-2.1.0 beta RC
>> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ or you could
>> check the same with trunk build.****
>>
>> ** **
>>
>> Thanks****
>>
>> Devaraj k****
>>
>> ** **
>>
>> *From:* Krishna Kishore Bonagiri [mailto:write2kishore@gmail.com]
>> *Sent:* 04 July 2013 21:33
>>
>> *To:* user@hadoop.apache.org
>> *Subject:* Re: Requesting containers on a specific host****
>>
>> ** **
>>
>> Thanks Arun, it seems to be available with 2.1.0-beta, when will that be
>> released? Or if I want it now, could I get from the trunk?****
>>
>> ** **
>>
>> -Kishore****
>>
>> ** **
>>
>> On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:****
>>
>> To guarantee nodes on a specific container you need to use the whitelist
>> feature we added recently:****
>>
>> https://issues.apache.org/jira/browse/YARN-398****
>>
>> Arun****
>>
>> ** **
>>
>> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
>> write2kishore@gmail.com> wrote:****
>>
>>
>>
>> ****
>>
>> I could get containers on specific nodes using addContainerRequest() on
>> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
>> in my cluster. And, my Application Master is trying to get 3 containers on
>> node1, and 3 containers on node2 in that order. ****
>>
>> ** **
>>
>> While trying to request on node1, it sometimes gives me those on node2,
>> and vice verse. When I get a container on a different node than the one I
>> need, I release it and make a fresh request. I am having to do like that
>> forever to get a container on the node I need. ****
>>
>> ** **
>>
>> Though the node I am requesting has enough resources, why does it keep
>> giving me containers on the other node? How can I make sure I get a
>> container on the node I want? ****
>>
>> ** **
>>
>> Note: I am using the default scheduler, i.e. Capacity Scheduler.****
>>
>> ** **
>>
>> Thanks,****
>>
>> Kishore****
>>
>> ** **
>>
>> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:****
>>
>> Check if the hostname you are setting is the same in the RM logs…****
>>
>> ** **
>>
>> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
>> write2kishore@gmail.com> wrote:****
>>
>>
>>
>> ****
>>
>> Hi,****
>>
>> I am trying to get container on a specific host, using setHostName(0
>> call on ResourceRequest, but could not get allocated anything forever,
>> which just works fine when I change the node name to "*". I am working on a
>> single node cluster, and I am giving the name of the single node I have in
>> my cluster. ****
>>
>> ** **
>>
>> Is there any specific format that I need to give for setHostName(), why
>> is it not working...****
>>
>> ** **
>>
>> Thanks,****
>>
>> Kishore****
>>
>> ** **
>>
>> --****
>>
>> Arun C. Murthy****
>>
>> Hortonworks Inc.
>> http://hortonworks.com/****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> --****
>>
>> Arun C. Murthy****
>>
>> Hortonworks Inc.
>> http://hortonworks.com/****
>>
>> ** **
>>
>> ** **
>>
>
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
I could resolve this error with a simple host name change to a fully
qualified one including the domain name. But to go ahead running some of my
old example code, I am observing at least the following changes :
1) ClientRMProtocol and AMRMProtocol are removed.
2) ContainerManager is removed.
3) YarnRemoteException is removed.
4) ContainerRequest is removed.
It looks like it is now compulsory to modify my application accordingly,
where as on earlier versions, I could have run the same application without
any modifications.
Thanks,
Kishore
On Fri, Jul 5, 2013 at 7:37 PM, Krishna Kishore Bonagiri <
write2kishore@gmail.com> wrote:
> Hi Devaraj,
>
> Thanks for pointing me to this RC. I am trying this out, and getting
> this error for NM to get started. My RM is running fine, but NM is failing
> and saying that it is disallowed by RM and received a SHUTDOWN message.
> Please give me clue to resolve this.
>
> 2013-07-05 09:49:20,043 ERROR
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Unexpected
> error starting NodeStatusUpdater
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN
> signal from Resourcemanager ,Registration of NodeManager failed, Message
> from ResourceManager: Disallowed NodeManager from
> isredeng.swg.usma.ibm.com, Sending SHUTDOWN signal to the NodeManager.
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:290)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:156)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:401)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:447)
>
> Thanks,
> Kishore
>
>
> On Fri, Jul 5, 2013 at 8:52 AM, Devaraj k <de...@huawei.com> wrote:
>
>> Hi Kishore,****
>>
>> ** **
>>
>> hadoop-2.1.0 beta release is in voting process now.****
>>
>> ** **
>>
>> You can try out from hadoop-2.1.0 beta RC
>> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ or you could
>> check the same with trunk build.****
>>
>> ** **
>>
>> Thanks****
>>
>> Devaraj k****
>>
>> ** **
>>
>> *From:* Krishna Kishore Bonagiri [mailto:write2kishore@gmail.com]
>> *Sent:* 04 July 2013 21:33
>>
>> *To:* user@hadoop.apache.org
>> *Subject:* Re: Requesting containers on a specific host****
>>
>> ** **
>>
>> Thanks Arun, it seems to be available with 2.1.0-beta, when will that be
>> released? Or if I want it now, could I get from the trunk?****
>>
>> ** **
>>
>> -Kishore****
>>
>> ** **
>>
>> On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:****
>>
>> To guarantee nodes on a specific container you need to use the whitelist
>> feature we added recently:****
>>
>> https://issues.apache.org/jira/browse/YARN-398****
>>
>> Arun****
>>
>> ** **
>>
>> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
>> write2kishore@gmail.com> wrote:****
>>
>>
>>
>> ****
>>
>> I could get containers on specific nodes using addContainerRequest() on
>> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
>> in my cluster. And, my Application Master is trying to get 3 containers on
>> node1, and 3 containers on node2 in that order. ****
>>
>> ** **
>>
>> While trying to request on node1, it sometimes gives me those on node2,
>> and vice verse. When I get a container on a different node than the one I
>> need, I release it and make a fresh request. I am having to do like that
>> forever to get a container on the node I need. ****
>>
>> ** **
>>
>> Though the node I am requesting has enough resources, why does it keep
>> giving me containers on the other node? How can I make sure I get a
>> container on the node I want? ****
>>
>> ** **
>>
>> Note: I am using the default scheduler, i.e. Capacity Scheduler.****
>>
>> ** **
>>
>> Thanks,****
>>
>> Kishore****
>>
>> ** **
>>
>> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:****
>>
>> Check if the hostname you are setting is the same in the RM logs…****
>>
>> ** **
>>
>> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
>> write2kishore@gmail.com> wrote:****
>>
>>
>>
>> ****
>>
>> Hi,****
>>
>> I am trying to get container on a specific host, using setHostName(0
>> call on ResourceRequest, but could not get allocated anything forever,
>> which just works fine when I change the node name to "*". I am working on a
>> single node cluster, and I am giving the name of the single node I have in
>> my cluster. ****
>>
>> ** **
>>
>> Is there any specific format that I need to give for setHostName(), why
>> is it not working...****
>>
>> ** **
>>
>> Thanks,****
>>
>> Kishore****
>>
>> ** **
>>
>> --****
>>
>> Arun C. Murthy****
>>
>> Hortonworks Inc.
>> http://hortonworks.com/****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> --****
>>
>> Arun C. Murthy****
>>
>> Hortonworks Inc.
>> http://hortonworks.com/****
>>
>> ** **
>>
>> ** **
>>
>
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Hi Devaraj,
Thanks for pointing me to this RC. I am trying this out, and getting
this error for NM to get started. My RM is running fine, but NM is failing
and saying that it is disallowed by RM and received a SHUTDOWN message.
Please give me clue to resolve this.
2013-07-05 09:49:20,043 ERROR
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Unexpected
error starting NodeStatusUpdater
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN
signal from Resourcemanager ,Registration of NodeManager failed, Message
from ResourceManager: Disallowed NodeManager from isredeng.swg.usma.ibm.com,
Sending SHUTDOWN signal to the NodeManager.
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:290)
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:156)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:401)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:447)
Thanks,
Kishore
On Fri, Jul 5, 2013 at 8:52 AM, Devaraj k <de...@huawei.com> wrote:
> Hi Kishore,****
>
> ** **
>
> hadoop-2.1.0 beta release is in voting process now.****
>
> ** **
>
> You can try out from hadoop-2.1.0 beta RC
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ or you could
> check the same with trunk build.****
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> ** **
>
> *From:* Krishna Kishore Bonagiri [mailto:write2kishore@gmail.com]
> *Sent:* 04 July 2013 21:33
>
> *To:* user@hadoop.apache.org
> *Subject:* Re: Requesting containers on a specific host****
>
> ** **
>
> Thanks Arun, it seems to be available with 2.1.0-beta, when will that be
> released? Or if I want it now, could I get from the trunk?****
>
> ** **
>
> -Kishore****
>
> ** **
>
> On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> ****
>
> To guarantee nodes on a specific container you need to use the whitelist
> feature we added recently:****
>
> https://issues.apache.org/jira/browse/YARN-398****
>
> Arun****
>
> ** **
>
> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> I could get containers on specific nodes using addContainerRequest() on
> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
> in my cluster. And, my Application Master is trying to get 3 containers on
> node1, and 3 containers on node2 in that order. ****
>
> ** **
>
> While trying to request on node1, it sometimes gives me those on node2,
> and vice verse. When I get a container on a different node than the one I
> need, I release it and make a fresh request. I am having to do like that
> forever to get a container on the node I need. ****
>
> ** **
>
> Though the node I am requesting has enough resources, why does it keep
> giving me containers on the other node? How can I make sure I get a
> container on the node I want? ****
>
> ** **
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>
> wrote:****
>
> Check if the hostname you are setting is the same in the RM logs…****
>
> ** **
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> Hi,****
>
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster. ****
>
> ** **
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
> ** **
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
> ** **
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Hi Devaraj,
Thanks for pointing me to this RC. I am trying this out, and getting
this error for NM to get started. My RM is running fine, but NM is failing
and saying that it is disallowed by RM and received a SHUTDOWN message.
Please give me clue to resolve this.
2013-07-05 09:49:20,043 ERROR
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Unexpected
error starting NodeStatusUpdater
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN
signal from Resourcemanager ,Registration of NodeManager failed, Message
from ResourceManager: Disallowed NodeManager from isredeng.swg.usma.ibm.com,
Sending SHUTDOWN signal to the NodeManager.
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:290)
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:156)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:401)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:447)
Thanks,
Kishore
On Fri, Jul 5, 2013 at 8:52 AM, Devaraj k <de...@huawei.com> wrote:
> Hi Kishore,****
>
> ** **
>
> hadoop-2.1.0 beta release is in voting process now.****
>
> ** **
>
> You can try out from hadoop-2.1.0 beta RC
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ or you could
> check the same with trunk build.****
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> ** **
>
> *From:* Krishna Kishore Bonagiri [mailto:write2kishore@gmail.com]
> *Sent:* 04 July 2013 21:33
>
> *To:* user@hadoop.apache.org
> *Subject:* Re: Requesting containers on a specific host****
>
> ** **
>
> Thanks Arun, it seems to be available with 2.1.0-beta, when will that be
> released? Or if I want it now, could I get from the trunk?****
>
> ** **
>
> -Kishore****
>
> ** **
>
> On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> ****
>
> To guarantee nodes on a specific container you need to use the whitelist
> feature we added recently:****
>
> https://issues.apache.org/jira/browse/YARN-398****
>
> Arun****
>
> ** **
>
> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> I could get containers on specific nodes using addContainerRequest() on
> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
> in my cluster. And, my Application Master is trying to get 3 containers on
> node1, and 3 containers on node2 in that order. ****
>
> ** **
>
> While trying to request on node1, it sometimes gives me those on node2,
> and vice verse. When I get a container on a different node than the one I
> need, I release it and make a fresh request. I am having to do like that
> forever to get a container on the node I need. ****
>
> ** **
>
> Though the node I am requesting has enough resources, why does it keep
> giving me containers on the other node? How can I make sure I get a
> container on the node I want? ****
>
> ** **
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>
> wrote:****
>
> Check if the hostname you are setting is the same in the RM logs…****
>
> ** **
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> Hi,****
>
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster. ****
>
> ** **
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
> ** **
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
> ** **
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Hi Devaraj,
Thanks for pointing me to this RC. I am trying this out, and getting
this error for NM to get started. My RM is running fine, but NM is failing
and saying that it is disallowed by RM and received a SHUTDOWN message.
Please give me clue to resolve this.
2013-07-05 09:49:20,043 ERROR
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Unexpected
error starting NodeStatusUpdater
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN
signal from Resourcemanager ,Registration of NodeManager failed, Message
from ResourceManager: Disallowed NodeManager from isredeng.swg.usma.ibm.com,
Sending SHUTDOWN signal to the NodeManager.
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:290)
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:156)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:401)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:447)
Thanks,
Kishore
On Fri, Jul 5, 2013 at 8:52 AM, Devaraj k <de...@huawei.com> wrote:
> Hi Kishore,****
>
> ** **
>
> hadoop-2.1.0 beta release is in voting process now.****
>
> ** **
>
> You can try out from hadoop-2.1.0 beta RC
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ or you could
> check the same with trunk build.****
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> ** **
>
> *From:* Krishna Kishore Bonagiri [mailto:write2kishore@gmail.com]
> *Sent:* 04 July 2013 21:33
>
> *To:* user@hadoop.apache.org
> *Subject:* Re: Requesting containers on a specific host****
>
> ** **
>
> Thanks Arun, it seems to be available with 2.1.0-beta, when will that be
> released? Or if I want it now, could I get from the trunk?****
>
> ** **
>
> -Kishore****
>
> ** **
>
> On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> ****
>
> To guarantee nodes on a specific container you need to use the whitelist
> feature we added recently:****
>
> https://issues.apache.org/jira/browse/YARN-398****
>
> Arun****
>
> ** **
>
> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> I could get containers on specific nodes using addContainerRequest() on
> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
> in my cluster. And, my Application Master is trying to get 3 containers on
> node1, and 3 containers on node2 in that order. ****
>
> ** **
>
> While trying to request on node1, it sometimes gives me those on node2,
> and vice verse. When I get a container on a different node than the one I
> need, I release it and make a fresh request. I am having to do like that
> forever to get a container on the node I need. ****
>
> ** **
>
> Though the node I am requesting has enough resources, why does it keep
> giving me containers on the other node? How can I make sure I get a
> container on the node I want? ****
>
> ** **
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>
> wrote:****
>
> Check if the hostname you are setting is the same in the RM logs…****
>
> ** **
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> Hi,****
>
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster. ****
>
> ** **
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
> ** **
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
> ** **
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Hi Devaraj,
Thanks for pointing me to this RC. I am trying this out, and getting
this error for NM to get started. My RM is running fine, but NM is failing
and saying that it is disallowed by RM and received a SHUTDOWN message.
Please give me clue to resolve this.
2013-07-05 09:49:20,043 ERROR
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Unexpected
error starting NodeStatusUpdater
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN
signal from Resourcemanager ,Registration of NodeManager failed, Message
from ResourceManager: Disallowed NodeManager from isredeng.swg.usma.ibm.com,
Sending SHUTDOWN signal to the NodeManager.
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:290)
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:156)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:401)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:447)
Thanks,
Kishore
On Fri, Jul 5, 2013 at 8:52 AM, Devaraj k <de...@huawei.com> wrote:
> Hi Kishore,****
>
> ** **
>
> hadoop-2.1.0 beta release is in voting process now.****
>
> ** **
>
> You can try out from hadoop-2.1.0 beta RC
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ or you could
> check the same with trunk build.****
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> ** **
>
> *From:* Krishna Kishore Bonagiri [mailto:write2kishore@gmail.com]
> *Sent:* 04 July 2013 21:33
>
> *To:* user@hadoop.apache.org
> *Subject:* Re: Requesting containers on a specific host****
>
> ** **
>
> Thanks Arun, it seems to be available with 2.1.0-beta, when will that be
> released? Or if I want it now, could I get from the trunk?****
>
> ** **
>
> -Kishore****
>
> ** **
>
> On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> ****
>
> To guarantee nodes on a specific container you need to use the whitelist
> feature we added recently:****
>
> https://issues.apache.org/jira/browse/YARN-398****
>
> Arun****
>
> ** **
>
> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> I could get containers on specific nodes using addContainerRequest() on
> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
> in my cluster. And, my Application Master is trying to get 3 containers on
> node1, and 3 containers on node2 in that order. ****
>
> ** **
>
> While trying to request on node1, it sometimes gives me those on node2,
> and vice verse. When I get a container on a different node than the one I
> need, I release it and make a fresh request. I am having to do like that
> forever to get a container on the node I need. ****
>
> ** **
>
> Though the node I am requesting has enough resources, why does it keep
> giving me containers on the other node? How can I make sure I get a
> container on the node I want? ****
>
> ** **
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>
> wrote:****
>
> Check if the hostname you are setting is the same in the RM logs…****
>
> ** **
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> Hi,****
>
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster. ****
>
> ** **
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
> ** **
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
> ** **
>
RE: Requesting containers on a specific host
Posted by Devaraj k <de...@huawei.com>.
Hi Kishore,
hadoop-2.1.0 beta release is in voting process now.
You can try out from hadoop-2.1.0 beta RC http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ or you could check the same with trunk build.
Thanks
Devaraj k
From: Krishna Kishore Bonagiri [mailto:write2kishore@gmail.com]
Sent: 04 July 2013 21:33
To: user@hadoop.apache.org
Subject: Re: Requesting containers on a specific host
Thanks Arun, it seems to be available with 2.1.0-beta, when will that be released? Or if I want it now, could I get from the trunk?
-Kishore
On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com>> wrote:
To guarantee nodes on a specific container you need to use the whitelist feature we added recently:
https://issues.apache.org/jira/browse/YARN-398
Arun
On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
I could get containers on specific nodes using addContainerRequest() on AMRMClient. But there are issues with it. I have two nodes, node1 and node2 in my cluster. And, my Application Master is trying to get 3 containers on node1, and 3 containers on node2 in that order.
While trying to request on node1, it sometimes gives me those on node2, and vice verse. When I get a container on a different node than the one I need, I release it and make a fresh request. I am having to do like that forever to get a container on the node I need.
Though the node I am requesting has enough resources, why does it keep giving me containers on the other node? How can I make sure I get a container on the node I want?
Note: I am using the default scheduler, i.e. Capacity Scheduler.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>> wrote:
Check if the hostname you are setting is the same in the RM logs...
On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
Hi,
I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
Is there any specific format that I need to give for setHostName(), why is it not working...
Thanks,
Kishore
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
RE: Requesting containers on a specific host
Posted by Devaraj k <de...@huawei.com>.
Hi Kishore,
hadoop-2.1.0 beta release is in voting process now.
You can try out from hadoop-2.1.0 beta RC http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ or you could check the same with trunk build.
Thanks
Devaraj k
From: Krishna Kishore Bonagiri [mailto:write2kishore@gmail.com]
Sent: 04 July 2013 21:33
To: user@hadoop.apache.org
Subject: Re: Requesting containers on a specific host
Thanks Arun, it seems to be available with 2.1.0-beta, when will that be released? Or if I want it now, could I get from the trunk?
-Kishore
On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com>> wrote:
To guarantee nodes on a specific container you need to use the whitelist feature we added recently:
https://issues.apache.org/jira/browse/YARN-398
Arun
On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
I could get containers on specific nodes using addContainerRequest() on AMRMClient. But there are issues with it. I have two nodes, node1 and node2 in my cluster. And, my Application Master is trying to get 3 containers on node1, and 3 containers on node2 in that order.
While trying to request on node1, it sometimes gives me those on node2, and vice verse. When I get a container on a different node than the one I need, I release it and make a fresh request. I am having to do like that forever to get a container on the node I need.
Though the node I am requesting has enough resources, why does it keep giving me containers on the other node? How can I make sure I get a container on the node I want?
Note: I am using the default scheduler, i.e. Capacity Scheduler.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>> wrote:
Check if the hostname you are setting is the same in the RM logs...
On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
Hi,
I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
Is there any specific format that I need to give for setHostName(), why is it not working...
Thanks,
Kishore
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
RE: Requesting containers on a specific host
Posted by Devaraj k <de...@huawei.com>.
Hi Kishore,
hadoop-2.1.0 beta release is in voting process now.
You can try out from hadoop-2.1.0 beta RC http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ or you could check the same with trunk build.
Thanks
Devaraj k
From: Krishna Kishore Bonagiri [mailto:write2kishore@gmail.com]
Sent: 04 July 2013 21:33
To: user@hadoop.apache.org
Subject: Re: Requesting containers on a specific host
Thanks Arun, it seems to be available with 2.1.0-beta, when will that be released? Or if I want it now, could I get from the trunk?
-Kishore
On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com>> wrote:
To guarantee nodes on a specific container you need to use the whitelist feature we added recently:
https://issues.apache.org/jira/browse/YARN-398
Arun
On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
I could get containers on specific nodes using addContainerRequest() on AMRMClient. But there are issues with it. I have two nodes, node1 and node2 in my cluster. And, my Application Master is trying to get 3 containers on node1, and 3 containers on node2 in that order.
While trying to request on node1, it sometimes gives me those on node2, and vice verse. When I get a container on a different node than the one I need, I release it and make a fresh request. I am having to do like that forever to get a container on the node I need.
Though the node I am requesting has enough resources, why does it keep giving me containers on the other node? How can I make sure I get a container on the node I want?
Note: I am using the default scheduler, i.e. Capacity Scheduler.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>> wrote:
Check if the hostname you are setting is the same in the RM logs...
On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
Hi,
I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
Is there any specific format that I need to give for setHostName(), why is it not working...
Thanks,
Kishore
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
RE: Requesting containers on a specific host
Posted by Devaraj k <de...@huawei.com>.
Hi Kishore,
hadoop-2.1.0 beta release is in voting process now.
You can try out from hadoop-2.1.0 beta RC http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ or you could check the same with trunk build.
Thanks
Devaraj k
From: Krishna Kishore Bonagiri [mailto:write2kishore@gmail.com]
Sent: 04 July 2013 21:33
To: user@hadoop.apache.org
Subject: Re: Requesting containers on a specific host
Thanks Arun, it seems to be available with 2.1.0-beta, when will that be released? Or if I want it now, could I get from the trunk?
-Kishore
On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com>> wrote:
To guarantee nodes on a specific container you need to use the whitelist feature we added recently:
https://issues.apache.org/jira/browse/YARN-398
Arun
On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
I could get containers on specific nodes using addContainerRequest() on AMRMClient. But there are issues with it. I have two nodes, node1 and node2 in my cluster. And, my Application Master is trying to get 3 containers on node1, and 3 containers on node2 in that order.
While trying to request on node1, it sometimes gives me those on node2, and vice verse. When I get a container on a different node than the one I need, I release it and make a fresh request. I am having to do like that forever to get a container on the node I need.
Though the node I am requesting has enough resources, why does it keep giving me containers on the other node? How can I make sure I get a container on the node I want?
Note: I am using the default scheduler, i.e. Capacity Scheduler.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>> wrote:
Check if the hostname you are setting is the same in the RM logs...
On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
Hi,
I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
Is there any specific format that I need to give for setHostName(), why is it not working...
Thanks,
Kishore
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Thanks Arun, it seems to be available with 2.1.0-beta, when will that be
released? Or if I want it now, could I get from the trunk?
-Kishore
On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> To guarantee nodes on a specific container you need to use the whitelist
> feature we added recently:
>
> https://issues.apache.org/jira/browse/YARN-398
>
> Arun
>
> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> I could get containers on specific nodes using addContainerRequest() on
> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
> in my cluster. And, my Application Master is trying to get 3 containers on
> node1, and 3 containers on node2 in that order.
>
> While trying to request on node1, it sometimes gives me those on node2,
> and vice verse. When I get a container on a different node than the one I
> need, I release it and make a fresh request. I am having to do like that
> forever to get a container on the node I need.
>
> Though the node I am requesting has enough resources, why does it keep
> giving me containers on the other node? How can I make sure I get a
> container on the node I want?
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.
>
> Thanks,
> Kishore
>
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>wrote:
>
>> Check if the hostname you are setting is the same in the RM logs…
>>
>> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
>> write2kishore@gmail.com> wrote:
>>
>> Hi,
>> I am trying to get container on a specific host, using setHostName(0
>> call on ResourceRequest, but could not get allocated anything forever,
>> which just works fine when I change the node name to "*". I am working on a
>> single node cluster, and I am giving the name of the single node I have in
>> my cluster.
>>
>> Is there any specific format that I need to give for setHostName(), why
>> is it not working...
>>
>> Thanks,
>> Kishore
>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Thanks Arun, it seems to be available with 2.1.0-beta, when will that be
released? Or if I want it now, could I get from the trunk?
-Kishore
On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> To guarantee nodes on a specific container you need to use the whitelist
> feature we added recently:
>
> https://issues.apache.org/jira/browse/YARN-398
>
> Arun
>
> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> I could get containers on specific nodes using addContainerRequest() on
> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
> in my cluster. And, my Application Master is trying to get 3 containers on
> node1, and 3 containers on node2 in that order.
>
> While trying to request on node1, it sometimes gives me those on node2,
> and vice verse. When I get a container on a different node than the one I
> need, I release it and make a fresh request. I am having to do like that
> forever to get a container on the node I need.
>
> Though the node I am requesting has enough resources, why does it keep
> giving me containers on the other node? How can I make sure I get a
> container on the node I want?
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.
>
> Thanks,
> Kishore
>
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>wrote:
>
>> Check if the hostname you are setting is the same in the RM logs…
>>
>> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
>> write2kishore@gmail.com> wrote:
>>
>> Hi,
>> I am trying to get container on a specific host, using setHostName(0
>> call on ResourceRequest, but could not get allocated anything forever,
>> which just works fine when I change the node name to "*". I am working on a
>> single node cluster, and I am giving the name of the single node I have in
>> my cluster.
>>
>> Is there any specific format that I need to give for setHostName(), why
>> is it not working...
>>
>> Thanks,
>> Kishore
>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: Requesting containers on a specific host
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
John,
This feature will available in the upcoming 2.1.0-beta release. The first
release candidate (RC) has been cut, but it seems a new RC will be needed.
The exact release date is still not known but it should be soon.
thanks.
On Thu, Jul 4, 2013 at 2:43 PM, John Lilley <jo...@redpoint.net>wrote:
> Arun,****
>
> I’m don’t know how to interpret the release schedule from the JIRA. It
> says that the patch targets 2.1.0 and it is checked into the trunk, does
> that mean it is likely to be rolled into the first Hadoop 2 GA or will it
> have to wait for another cycle?****
>
> Thanks,****
>
> John****
>
> ** **
>
> *From:* Arun C Murthy [mailto:acm@hortonworks.com]
> *Sent:* Thursday, July 04, 2013 6:28 AM
> *To:* user@hadoop.apache.org
> *Subject:* Re: Requesting containers on a specific host****
>
> ** **
>
> To guarantee nodes on a specific container you need to use the whitelist
> feature we added recently:****
>
> https://issues.apache.org/jira/browse/YARN-398****
>
> Arun****
>
> ** **
>
> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> I could get containers on specific nodes using addContainerRequest() on
> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
> in my cluster. And, my Application Master is trying to get 3 containers on
> node1, and 3 containers on node2 in that order. ****
>
> ** **
>
> While trying to request on node1, it sometimes gives me those on node2,
> and vice verse. When I get a container on a different node than the one I
> need, I release it and make a fresh request. I am having to do like that
> forever to get a container on the node I need. ****
>
> ** **
>
> Though the node I am requesting has enough resources, why does it keep
> giving me containers on the other node? How can I make sure I get a
> container on the node I want? ****
>
> ** **
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>
> wrote:****
>
> Check if the hostname you are setting is the same in the RM logs…****
>
> ** **
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> Hi,****
>
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster. ****
>
> ** **
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
> ** **
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
--
Alejandro
Re: Requesting containers on a specific host
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
John,
This feature will available in the upcoming 2.1.0-beta release. The first
release candidate (RC) has been cut, but it seems a new RC will be needed.
The exact release date is still not known but it should be soon.
thanks.
On Thu, Jul 4, 2013 at 2:43 PM, John Lilley <jo...@redpoint.net>wrote:
> Arun,****
>
> I’m don’t know how to interpret the release schedule from the JIRA. It
> says that the patch targets 2.1.0 and it is checked into the trunk, does
> that mean it is likely to be rolled into the first Hadoop 2 GA or will it
> have to wait for another cycle?****
>
> Thanks,****
>
> John****
>
> ** **
>
> *From:* Arun C Murthy [mailto:acm@hortonworks.com]
> *Sent:* Thursday, July 04, 2013 6:28 AM
> *To:* user@hadoop.apache.org
> *Subject:* Re: Requesting containers on a specific host****
>
> ** **
>
> To guarantee nodes on a specific container you need to use the whitelist
> feature we added recently:****
>
> https://issues.apache.org/jira/browse/YARN-398****
>
> Arun****
>
> ** **
>
> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> I could get containers on specific nodes using addContainerRequest() on
> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
> in my cluster. And, my Application Master is trying to get 3 containers on
> node1, and 3 containers on node2 in that order. ****
>
> ** **
>
> While trying to request on node1, it sometimes gives me those on node2,
> and vice verse. When I get a container on a different node than the one I
> need, I release it and make a fresh request. I am having to do like that
> forever to get a container on the node I need. ****
>
> ** **
>
> Though the node I am requesting has enough resources, why does it keep
> giving me containers on the other node? How can I make sure I get a
> container on the node I want? ****
>
> ** **
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>
> wrote:****
>
> Check if the hostname you are setting is the same in the RM logs…****
>
> ** **
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> Hi,****
>
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster. ****
>
> ** **
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
> ** **
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
--
Alejandro
Re: Requesting containers on a specific host
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
John,
This feature will available in the upcoming 2.1.0-beta release. The first
release candidate (RC) has been cut, but it seems a new RC will be needed.
The exact release date is still not known but it should be soon.
thanks.
On Thu, Jul 4, 2013 at 2:43 PM, John Lilley <jo...@redpoint.net>wrote:
> Arun,****
>
> I’m don’t know how to interpret the release schedule from the JIRA. It
> says that the patch targets 2.1.0 and it is checked into the trunk, does
> that mean it is likely to be rolled into the first Hadoop 2 GA or will it
> have to wait for another cycle?****
>
> Thanks,****
>
> John****
>
> ** **
>
> *From:* Arun C Murthy [mailto:acm@hortonworks.com]
> *Sent:* Thursday, July 04, 2013 6:28 AM
> *To:* user@hadoop.apache.org
> *Subject:* Re: Requesting containers on a specific host****
>
> ** **
>
> To guarantee nodes on a specific container you need to use the whitelist
> feature we added recently:****
>
> https://issues.apache.org/jira/browse/YARN-398****
>
> Arun****
>
> ** **
>
> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> I could get containers on specific nodes using addContainerRequest() on
> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
> in my cluster. And, my Application Master is trying to get 3 containers on
> node1, and 3 containers on node2 in that order. ****
>
> ** **
>
> While trying to request on node1, it sometimes gives me those on node2,
> and vice verse. When I get a container on a different node than the one I
> need, I release it and make a fresh request. I am having to do like that
> forever to get a container on the node I need. ****
>
> ** **
>
> Though the node I am requesting has enough resources, why does it keep
> giving me containers on the other node? How can I make sure I get a
> container on the node I want? ****
>
> ** **
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>
> wrote:****
>
> Check if the hostname you are setting is the same in the RM logs…****
>
> ** **
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> Hi,****
>
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster. ****
>
> ** **
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
> ** **
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
--
Alejandro
Re: Requesting containers on a specific host
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
John,
This feature will available in the upcoming 2.1.0-beta release. The first
release candidate (RC) has been cut, but it seems a new RC will be needed.
The exact release date is still not known but it should be soon.
thanks.
On Thu, Jul 4, 2013 at 2:43 PM, John Lilley <jo...@redpoint.net>wrote:
> Arun,****
>
> I’m don’t know how to interpret the release schedule from the JIRA. It
> says that the patch targets 2.1.0 and it is checked into the trunk, does
> that mean it is likely to be rolled into the first Hadoop 2 GA or will it
> have to wait for another cycle?****
>
> Thanks,****
>
> John****
>
> ** **
>
> *From:* Arun C Murthy [mailto:acm@hortonworks.com]
> *Sent:* Thursday, July 04, 2013 6:28 AM
> *To:* user@hadoop.apache.org
> *Subject:* Re: Requesting containers on a specific host****
>
> ** **
>
> To guarantee nodes on a specific container you need to use the whitelist
> feature we added recently:****
>
> https://issues.apache.org/jira/browse/YARN-398****
>
> Arun****
>
> ** **
>
> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> I could get containers on specific nodes using addContainerRequest() on
> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
> in my cluster. And, my Application Master is trying to get 3 containers on
> node1, and 3 containers on node2 in that order. ****
>
> ** **
>
> While trying to request on node1, it sometimes gives me those on node2,
> and vice verse. When I get a container on a different node than the one I
> need, I release it and make a fresh request. I am having to do like that
> forever to get a container on the node I need. ****
>
> ** **
>
> Though the node I am requesting has enough resources, why does it keep
> giving me containers on the other node? How can I make sure I get a
> container on the node I want? ****
>
> ** **
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>
> wrote:****
>
> Check if the hostname you are setting is the same in the RM logs…****
>
> ** **
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:****
>
>
>
> ****
>
> Hi,****
>
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster. ****
>
> ** **
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...****
>
> ** **
>
> Thanks,****
>
> Kishore****
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
> ** **
>
> ** **
>
> --****
>
> Arun C. Murthy****
>
> Hortonworks Inc.
> http://hortonworks.com/****
>
> ** **
>
--
Alejandro
RE: Requesting containers on a specific host
Posted by John Lilley <jo...@redpoint.net>.
Arun,
I'm don't know how to interpret the release schedule from the JIRA. It says that the patch targets 2.1.0 and it is checked into the trunk, does that mean it is likely to be rolled into the first Hadoop 2 GA or will it have to wait for another cycle?
Thanks,
John
From: Arun C Murthy [mailto:acm@hortonworks.com]
Sent: Thursday, July 04, 2013 6:28 AM
To: user@hadoop.apache.org
Subject: Re: Requesting containers on a specific host
To guarantee nodes on a specific container you need to use the whitelist feature we added recently:
https://issues.apache.org/jira/browse/YARN-398
Arun
On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
I could get containers on specific nodes using addContainerRequest() on AMRMClient. But there are issues with it. I have two nodes, node1 and node2 in my cluster. And, my Application Master is trying to get 3 containers on node1, and 3 containers on node2 in that order.
While trying to request on node1, it sometimes gives me those on node2, and vice verse. When I get a container on a different node than the one I need, I release it and make a fresh request. I am having to do like that forever to get a container on the node I need.
Though the node I am requesting has enough resources, why does it keep giving me containers on the other node? How can I make sure I get a container on the node I want?
Note: I am using the default scheduler, i.e. Capacity Scheduler.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>> wrote:
Check if the hostname you are setting is the same in the RM logs...
On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
Hi,
I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
Is there any specific format that I need to give for setHostName(), why is it not working...
Thanks,
Kishore
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
RE: Requesting containers on a specific host
Posted by John Lilley <jo...@redpoint.net>.
Arun,
I'm don't know how to interpret the release schedule from the JIRA. It says that the patch targets 2.1.0 and it is checked into the trunk, does that mean it is likely to be rolled into the first Hadoop 2 GA or will it have to wait for another cycle?
Thanks,
John
From: Arun C Murthy [mailto:acm@hortonworks.com]
Sent: Thursday, July 04, 2013 6:28 AM
To: user@hadoop.apache.org
Subject: Re: Requesting containers on a specific host
To guarantee nodes on a specific container you need to use the whitelist feature we added recently:
https://issues.apache.org/jira/browse/YARN-398
Arun
On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
I could get containers on specific nodes using addContainerRequest() on AMRMClient. But there are issues with it. I have two nodes, node1 and node2 in my cluster. And, my Application Master is trying to get 3 containers on node1, and 3 containers on node2 in that order.
While trying to request on node1, it sometimes gives me those on node2, and vice verse. When I get a container on a different node than the one I need, I release it and make a fresh request. I am having to do like that forever to get a container on the node I need.
Though the node I am requesting has enough resources, why does it keep giving me containers on the other node? How can I make sure I get a container on the node I want?
Note: I am using the default scheduler, i.e. Capacity Scheduler.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>> wrote:
Check if the hostname you are setting is the same in the RM logs...
On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
Hi,
I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
Is there any specific format that I need to give for setHostName(), why is it not working...
Thanks,
Kishore
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Thanks Arun, it seems to be available with 2.1.0-beta, when will that be
released? Or if I want it now, could I get from the trunk?
-Kishore
On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> To guarantee nodes on a specific container you need to use the whitelist
> feature we added recently:
>
> https://issues.apache.org/jira/browse/YARN-398
>
> Arun
>
> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> I could get containers on specific nodes using addContainerRequest() on
> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
> in my cluster. And, my Application Master is trying to get 3 containers on
> node1, and 3 containers on node2 in that order.
>
> While trying to request on node1, it sometimes gives me those on node2,
> and vice verse. When I get a container on a different node than the one I
> need, I release it and make a fresh request. I am having to do like that
> forever to get a container on the node I need.
>
> Though the node I am requesting has enough resources, why does it keep
> giving me containers on the other node? How can I make sure I get a
> container on the node I want?
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.
>
> Thanks,
> Kishore
>
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>wrote:
>
>> Check if the hostname you are setting is the same in the RM logs…
>>
>> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
>> write2kishore@gmail.com> wrote:
>>
>> Hi,
>> I am trying to get container on a specific host, using setHostName(0
>> call on ResourceRequest, but could not get allocated anything forever,
>> which just works fine when I change the node name to "*". I am working on a
>> single node cluster, and I am giving the name of the single node I have in
>> my cluster.
>>
>> Is there any specific format that I need to give for setHostName(), why
>> is it not working...
>>
>> Thanks,
>> Kishore
>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
RE: Requesting containers on a specific host
Posted by John Lilley <jo...@redpoint.net>.
Arun,
I'm don't know how to interpret the release schedule from the JIRA. It says that the patch targets 2.1.0 and it is checked into the trunk, does that mean it is likely to be rolled into the first Hadoop 2 GA or will it have to wait for another cycle?
Thanks,
John
From: Arun C Murthy [mailto:acm@hortonworks.com]
Sent: Thursday, July 04, 2013 6:28 AM
To: user@hadoop.apache.org
Subject: Re: Requesting containers on a specific host
To guarantee nodes on a specific container you need to use the whitelist feature we added recently:
https://issues.apache.org/jira/browse/YARN-398
Arun
On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
I could get containers on specific nodes using addContainerRequest() on AMRMClient. But there are issues with it. I have two nodes, node1 and node2 in my cluster. And, my Application Master is trying to get 3 containers on node1, and 3 containers on node2 in that order.
While trying to request on node1, it sometimes gives me those on node2, and vice verse. When I get a container on a different node than the one I need, I release it and make a fresh request. I am having to do like that forever to get a container on the node I need.
Though the node I am requesting has enough resources, why does it keep giving me containers on the other node? How can I make sure I get a container on the node I want?
Note: I am using the default scheduler, i.e. Capacity Scheduler.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>> wrote:
Check if the hostname you are setting is the same in the RM logs...
On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
Hi,
I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
Is there any specific format that I need to give for setHostName(), why is it not working...
Thanks,
Kishore
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Thanks Arun, it seems to be available with 2.1.0-beta, when will that be
released? Or if I want it now, could I get from the trunk?
-Kishore
On Thu, Jul 4, 2013 at 5:58 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> To guarantee nodes on a specific container you need to use the whitelist
> feature we added recently:
>
> https://issues.apache.org/jira/browse/YARN-398
>
> Arun
>
> On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> I could get containers on specific nodes using addContainerRequest() on
> AMRMClient. But there are issues with it. I have two nodes, node1 and node2
> in my cluster. And, my Application Master is trying to get 3 containers on
> node1, and 3 containers on node2 in that order.
>
> While trying to request on node1, it sometimes gives me those on node2,
> and vice verse. When I get a container on a different node than the one I
> need, I release it and make a fresh request. I am having to do like that
> forever to get a container on the node I need.
>
> Though the node I am requesting has enough resources, why does it keep
> giving me containers on the other node? How can I make sure I get a
> container on the node I want?
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.
>
> Thanks,
> Kishore
>
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>wrote:
>
>> Check if the hostname you are setting is the same in the RM logs…
>>
>> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
>> write2kishore@gmail.com> wrote:
>>
>> Hi,
>> I am trying to get container on a specific host, using setHostName(0
>> call on ResourceRequest, but could not get allocated anything forever,
>> which just works fine when I change the node name to "*". I am working on a
>> single node cluster, and I am giving the name of the single node I have in
>> my cluster.
>>
>> Is there any specific format that I need to give for setHostName(), why
>> is it not working...
>>
>> Thanks,
>> Kishore
>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
RE: Requesting containers on a specific host
Posted by John Lilley <jo...@redpoint.net>.
Arun,
I'm don't know how to interpret the release schedule from the JIRA. It says that the patch targets 2.1.0 and it is checked into the trunk, does that mean it is likely to be rolled into the first Hadoop 2 GA or will it have to wait for another cycle?
Thanks,
John
From: Arun C Murthy [mailto:acm@hortonworks.com]
Sent: Thursday, July 04, 2013 6:28 AM
To: user@hadoop.apache.org
Subject: Re: Requesting containers on a specific host
To guarantee nodes on a specific container you need to use the whitelist feature we added recently:
https://issues.apache.org/jira/browse/YARN-398
Arun
On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
I could get containers on specific nodes using addContainerRequest() on AMRMClient. But there are issues with it. I have two nodes, node1 and node2 in my cluster. And, my Application Master is trying to get 3 containers on node1, and 3 containers on node2 in that order.
While trying to request on node1, it sometimes gives me those on node2, and vice verse. When I get a container on a different node than the one I need, I release it and make a fresh request. I am having to do like that forever to get a container on the node I need.
Though the node I am requesting has enough resources, why does it keep giving me containers on the other node? How can I make sure I get a container on the node I want?
Note: I am using the default scheduler, i.e. Capacity Scheduler.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com>> wrote:
Check if the hostname you are setting is the same in the RM logs...
On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com>> wrote:
Hi,
I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
Is there any specific format that I need to give for setHostName(), why is it not working...
Thanks,
Kishore
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Requesting containers on a specific host
Posted by Arun C Murthy <ac...@hortonworks.com>.
To guarantee nodes on a specific container you need to use the whitelist feature we added recently:
https://issues.apache.org/jira/browse/YARN-398
Arun
On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <wr...@gmail.com> wrote:
> I could get containers on specific nodes using addContainerRequest() on AMRMClient. But there are issues with it. I have two nodes, node1 and node2 in my cluster. And, my Application Master is trying to get 3 containers on node1, and 3 containers on node2 in that order.
>
> While trying to request on node1, it sometimes gives me those on node2, and vice verse. When I get a container on a different node than the one I need, I release it and make a fresh request. I am having to do like that forever to get a container on the node I need.
>
> Though the node I am requesting has enough resources, why does it keep giving me containers on the other node? How can I make sure I get a container on the node I want?
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.
>
> Thanks,
> Kishore
>
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com> wrote:
>
>> Hi,
>> I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
>>
>> Is there any specific format that I need to give for setHostName(), why is it not working...
>>
>> Thanks,
>> Kishore
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Requesting containers on a specific host
Posted by Arun C Murthy <ac...@hortonworks.com>.
To guarantee nodes on a specific container you need to use the whitelist feature we added recently:
https://issues.apache.org/jira/browse/YARN-398
Arun
On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <wr...@gmail.com> wrote:
> I could get containers on specific nodes using addContainerRequest() on AMRMClient. But there are issues with it. I have two nodes, node1 and node2 in my cluster. And, my Application Master is trying to get 3 containers on node1, and 3 containers on node2 in that order.
>
> While trying to request on node1, it sometimes gives me those on node2, and vice verse. When I get a container on a different node than the one I need, I release it and make a fresh request. I am having to do like that forever to get a container on the node I need.
>
> Though the node I am requesting has enough resources, why does it keep giving me containers on the other node? How can I make sure I get a container on the node I want?
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.
>
> Thanks,
> Kishore
>
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com> wrote:
>
>> Hi,
>> I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
>>
>> Is there any specific format that I need to give for setHostName(), why is it not working...
>>
>> Thanks,
>> Kishore
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Requesting containers on a specific host
Posted by Arun C Murthy <ac...@hortonworks.com>.
To guarantee nodes on a specific container you need to use the whitelist feature we added recently:
https://issues.apache.org/jira/browse/YARN-398
Arun
On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <wr...@gmail.com> wrote:
> I could get containers on specific nodes using addContainerRequest() on AMRMClient. But there are issues with it. I have two nodes, node1 and node2 in my cluster. And, my Application Master is trying to get 3 containers on node1, and 3 containers on node2 in that order.
>
> While trying to request on node1, it sometimes gives me those on node2, and vice verse. When I get a container on a different node than the one I need, I release it and make a fresh request. I am having to do like that forever to get a container on the node I need.
>
> Though the node I am requesting has enough resources, why does it keep giving me containers on the other node? How can I make sure I get a container on the node I want?
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.
>
> Thanks,
> Kishore
>
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com> wrote:
>
>> Hi,
>> I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
>>
>> Is there any specific format that I need to give for setHostName(), why is it not working...
>>
>> Thanks,
>> Kishore
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Requesting containers on a specific host
Posted by Arun C Murthy <ac...@hortonworks.com>.
To guarantee nodes on a specific container you need to use the whitelist feature we added recently:
https://issues.apache.org/jira/browse/YARN-398
Arun
On Jul 4, 2013, at 3:14 AM, Krishna Kishore Bonagiri <wr...@gmail.com> wrote:
> I could get containers on specific nodes using addContainerRequest() on AMRMClient. But there are issues with it. I have two nodes, node1 and node2 in my cluster. And, my Application Master is trying to get 3 containers on node1, and 3 containers on node2 in that order.
>
> While trying to request on node1, it sometimes gives me those on node2, and vice verse. When I get a container on a different node than the one I need, I release it and make a fresh request. I am having to do like that forever to get a container on the node I need.
>
> Though the node I am requesting has enough resources, why does it keep giving me containers on the other node? How can I make sure I get a container on the node I want?
>
> Note: I am using the default scheduler, i.e. Capacity Scheduler.
>
> Thanks,
> Kishore
>
>
> On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com> wrote:
>
>> Hi,
>> I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
>>
>> Is there any specific format that I need to give for setHostName(), why is it not working...
>>
>> Thanks,
>> Kishore
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
I could get containers on specific nodes using addContainerRequest() on
AMRMClient. But there are issues with it. I have two nodes, node1 and node2
in my cluster. And, my Application Master is trying to get 3 containers on
node1, and 3 containers on node2 in that order.
While trying to request on node1, it sometimes gives me those on node2, and
vice verse. When I get a container on a different node than the one I need,
I release it and make a fresh request. I am having to do like that forever
to get a container on the node I need.
Though the node I am requesting has enough resources, why does it keep
giving me containers on the other node? How can I make sure I get a
container on the node I want?
Note: I am using the default scheduler, i.e. Capacity Scheduler.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> Hi,
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster.
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...
>
> Thanks,
> Kishore
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Hi Arun,
I just found that setHostName() doesn't work on 2.0.0-alpha and works on
2.0..4-alpha, I didn't check the intermediate versions. I have verified it
by starting the daemons of respective versions, and modifying the
ApplicationMaster.java in the distributed shell example, and running a date
command on it. Is this something already known? or have I been doing
something wrong?
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> Hi,
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster.
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...
>
> Thanks,
> Kishore
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
I could get containers on specific nodes using addContainerRequest() on
AMRMClient. But there are issues with it. I have two nodes, node1 and node2
in my cluster. And, my Application Master is trying to get 3 containers on
node1, and 3 containers on node2 in that order.
While trying to request on node1, it sometimes gives me those on node2, and
vice verse. When I get a container on a different node than the one I need,
I release it and make a fresh request. I am having to do like that forever
to get a container on the node I need.
Though the node I am requesting has enough resources, why does it keep
giving me containers on the other node? How can I make sure I get a
container on the node I want?
Note: I am using the default scheduler, i.e. Capacity Scheduler.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> Hi,
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster.
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...
>
> Thanks,
> Kishore
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
I could get containers on specific nodes using addContainerRequest() on
AMRMClient. But there are issues with it. I have two nodes, node1 and node2
in my cluster. And, my Application Master is trying to get 3 containers on
node1, and 3 containers on node2 in that order.
While trying to request on node1, it sometimes gives me those on node2, and
vice verse. When I get a container on a different node than the one I need,
I release it and make a fresh request. I am having to do like that forever
to get a container on the node I need.
Though the node I am requesting has enough resources, why does it keep
giving me containers on the other node? How can I make sure I get a
container on the node I want?
Note: I am using the default scheduler, i.e. Capacity Scheduler.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> Hi,
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster.
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...
>
> Thanks,
> Kishore
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Hi Arun,
I just found that setHostName() doesn't work on 2.0.0-alpha and works on
2.0..4-alpha, I didn't check the intermediate versions. I have verified it
by starting the daemons of respective versions, and modifying the
ApplicationMaster.java in the distributed shell example, and running a date
command on it. Is this something already known? or have I been doing
something wrong?
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> Hi,
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster.
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...
>
> Thanks,
> Kishore
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Yes Arun, the one I am giving is same as the one I see in RM's log also.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> Hi,
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster.
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...
>
> Thanks,
> Kishore
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Hi Arun,
I just found that setHostName() doesn't work on 2.0.0-alpha and works on
2.0..4-alpha, I didn't check the intermediate versions. I have verified it
by starting the daemons of respective versions, and modifying the
ApplicationMaster.java in the distributed shell example, and running a date
command on it. Is this something already known? or have I been doing
something wrong?
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> Hi,
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster.
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...
>
> Thanks,
> Kishore
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
I could get containers on specific nodes using addContainerRequest() on
AMRMClient. But there are issues with it. I have two nodes, node1 and node2
in my cluster. And, my Application Master is trying to get 3 containers on
node1, and 3 containers on node2 in that order.
While trying to request on node1, it sometimes gives me those on node2, and
vice verse. When I get a container on a different node than the one I need,
I release it and make a fresh request. I am having to do like that forever
to get a container on the node I need.
Though the node I am requesting has enough resources, why does it keep
giving me containers on the other node? How can I make sure I get a
container on the node I want?
Note: I am using the default scheduler, i.e. Capacity Scheduler.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> Hi,
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster.
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...
>
> Thanks,
> Kishore
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Yes Arun, the one I am giving is same as the one I see in RM's log also.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> Hi,
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster.
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...
>
> Thanks,
> Kishore
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: Requesting containers on a specific host
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Yes Arun, the one I am giving is same as the one I see in RM's log also.
Thanks,
Kishore
On Fri, Jun 21, 2013 at 7:25 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Check if the hostname you are setting is the same in the RM logs…
>
> On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> Hi,
> I am trying to get container on a specific host, using setHostName(0
> call on ResourceRequest, but could not get allocated anything forever,
> which just works fine when I change the node name to "*". I am working on a
> single node cluster, and I am giving the name of the single node I have in
> my cluster.
>
> Is there any specific format that I need to give for setHostName(), why
> is it not working...
>
> Thanks,
> Kishore
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: Requesting containers on a specific host
Posted by Arun C Murthy <ac...@hortonworks.com>.
Check if the hostname you are setting is the same in the RM logs…
On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com> wrote:
> Hi,
> I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
>
> Is there any specific format that I need to give for setHostName(), why is it not working...
>
> Thanks,
> Kishore
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Requesting containers on a specific host
Posted by Arun C Murthy <ac...@hortonworks.com>.
Check if the hostname you are setting is the same in the RM logs…
On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com> wrote:
> Hi,
> I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
>
> Is there any specific format that I need to give for setHostName(), why is it not working...
>
> Thanks,
> Kishore
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Requesting containers on a specific host
Posted by Arun C Murthy <ac...@hortonworks.com>.
Check if the hostname you are setting is the same in the RM logs…
On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com> wrote:
> Hi,
> I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
>
> Is there any specific format that I need to give for setHostName(), why is it not working...
>
> Thanks,
> Kishore
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Requesting containers on a specific host
Posted by Arun C Murthy <ac...@hortonworks.com>.
Check if the hostname you are setting is the same in the RM logs…
On Jun 21, 2013, at 2:15 AM, Krishna Kishore Bonagiri <wr...@gmail.com> wrote:
> Hi,
> I am trying to get container on a specific host, using setHostName(0 call on ResourceRequest, but could not get allocated anything forever, which just works fine when I change the node name to "*". I am working on a single node cluster, and I am giving the name of the single node I have in my cluster.
>
> Is there any specific format that I need to give for setHostName(), why is it not working...
>
> Thanks,
> Kishore
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/