You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Krishna Kishore Bonagiri <wr...@gmail.com> on 2013/04/03 17:05:25 UTC
getAllocatedContainers() is not returning when ran against 2.0.3-alpha
Hi,
I am running an application that had been written and working well with
the hadoop-2.0.0-alpha but when I am running the same against 2.0.3-alpha,
the getAllocatedContainers() method called on AMResponse is not returning
all the containers allocated sometimes. For example, I request for 10
containers and this method gives me only 9 containers sometimes, and when I
looked at the log of Resource Manager, the 10th container is also
allocated. It happens only sometimes randomly and works fine all other
times. If I send one more request for the remaining container to RM, it
could allocate that container. I am running only one application at a time,
but 1000s of them one after another.
Is this problem seen by anyone else? I have seen that there are
significant number of changes in the way ApplicationMaster.java is written
now for the Distributed Shell example. There is now AMRMClient instead
of AMRMProtocol. There is method addContainerRequest() which is not there
earlier, and there is ConatainerRequest class now which was ResourceRequest
earlier, etc. Is it adivsable to re-write my application in a similar way?
Thanks,
Kishore
Re: getAllocatedContainers() is not returning when ran against 2.0.3-alpha
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Hi Hitesh,
Thanks for the reply. I understand that it may not always allocate all
the requested containers in one go. After the actual request is sent, my
code repeatedly sends an empty container request and checks the responses
until all the requested containers are allocated.
But my main worry is, even though the RM's log is saying that all 10
requested containers are allocated, the getAllocatedContainers() method is
not returning me all of them, it returned only 9 surprisingly.
I shall file a jira for this, but please let me know soon if you have
any clue about this.
Thanks,
Kishore
On Wed, Apr 3, 2013 at 9:20 PM, Hitesh Shah <hi...@hortonworks.com> wrote:
> If I understand your question, you are expecting all the containers to be
> allocated in one go? Or are you seeing your application hang because it
> asked for 10 containers but it only received a total of 9 even after
> repeated calls to the RM?
>
> There is no guarantee that you will be allocated all your requested
> containers in a single allocate() call. The application is expected to keep
> on making allocate calls to the RM at regular intervals and the RM will
> give it any newly allocated containers in the period since the last
> allocate call.
>
> The AMRMClient is a library that should make it easier for someone to
> write an application master. Yes - it is the recommended approach. Please
> file jiras if you see use-cases that are not addressed by it when
> implementing your AM.
>
> thanks
> -- Hitesh
>
> On Apr 3, 2013, at 8:05 AM, Krishna Kishore Bonagiri wrote:
>
> > Hi,
> >
> > I am running an application that had been written and working well with
> the hadoop-2.0.0-alpha but when I am running the same against 2.0.3-alpha,
> the getAllocatedContainers() method called on AMResponse is not returning
> all the containers allocated sometimes. For example, I request for 10
> containers and this method gives me only 9 containers sometimes, and when I
> looked at the log of Resource Manager, the 10th container is also
> allocated. It happens only sometimes randomly and works fine all other
> times. If I send one more request for the remaining container to RM, it
> could allocate that container. I am running only one application at a time,
> but 1000s of them one after another.
> >
> > Is this problem seen by anyone else? I have seen that there are
> significant number of changes in the way ApplicationMaster.java is written
> now for the Distributed Shell example. There is now AMRMClient instead of
> AMRMProtocol. There is method addContainerRequest() which is not there
> earlier, and there is ConatainerRequest class now which was ResourceRequest
> earlier, etc. Is it adivsable to re-write my application in a similar way?
> >
> > Thanks,
> > Kishore
> >
> >
>
>
Re: getAllocatedContainers() is not returning when ran against 2.0.3-alpha
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Hi Hitesh,
Thanks for the reply. I understand that it may not always allocate all
the requested containers in one go. After the actual request is sent, my
code repeatedly sends an empty container request and checks the responses
until all the requested containers are allocated.
But my main worry is, even though the RM's log is saying that all 10
requested containers are allocated, the getAllocatedContainers() method is
not returning me all of them, it returned only 9 surprisingly.
I shall file a jira for this, but please let me know soon if you have
any clue about this.
Thanks,
Kishore
On Wed, Apr 3, 2013 at 9:20 PM, Hitesh Shah <hi...@hortonworks.com> wrote:
> If I understand your question, you are expecting all the containers to be
> allocated in one go? Or are you seeing your application hang because it
> asked for 10 containers but it only received a total of 9 even after
> repeated calls to the RM?
>
> There is no guarantee that you will be allocated all your requested
> containers in a single allocate() call. The application is expected to keep
> on making allocate calls to the RM at regular intervals and the RM will
> give it any newly allocated containers in the period since the last
> allocate call.
>
> The AMRMClient is a library that should make it easier for someone to
> write an application master. Yes - it is the recommended approach. Please
> file jiras if you see use-cases that are not addressed by it when
> implementing your AM.
>
> thanks
> -- Hitesh
>
> On Apr 3, 2013, at 8:05 AM, Krishna Kishore Bonagiri wrote:
>
> > Hi,
> >
> > I am running an application that had been written and working well with
> the hadoop-2.0.0-alpha but when I am running the same against 2.0.3-alpha,
> the getAllocatedContainers() method called on AMResponse is not returning
> all the containers allocated sometimes. For example, I request for 10
> containers and this method gives me only 9 containers sometimes, and when I
> looked at the log of Resource Manager, the 10th container is also
> allocated. It happens only sometimes randomly and works fine all other
> times. If I send one more request for the remaining container to RM, it
> could allocate that container. I am running only one application at a time,
> but 1000s of them one after another.
> >
> > Is this problem seen by anyone else? I have seen that there are
> significant number of changes in the way ApplicationMaster.java is written
> now for the Distributed Shell example. There is now AMRMClient instead of
> AMRMProtocol. There is method addContainerRequest() which is not there
> earlier, and there is ConatainerRequest class now which was ResourceRequest
> earlier, etc. Is it adivsable to re-write my application in a similar way?
> >
> > Thanks,
> > Kishore
> >
> >
>
>
Re: getAllocatedContainers() is not returning when ran against 2.0.3-alpha
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Hi Hitesh,
Thanks for the reply. I understand that it may not always allocate all
the requested containers in one go. After the actual request is sent, my
code repeatedly sends an empty container request and checks the responses
until all the requested containers are allocated.
But my main worry is, even though the RM's log is saying that all 10
requested containers are allocated, the getAllocatedContainers() method is
not returning me all of them, it returned only 9 surprisingly.
I shall file a jira for this, but please let me know soon if you have
any clue about this.
Thanks,
Kishore
On Wed, Apr 3, 2013 at 9:20 PM, Hitesh Shah <hi...@hortonworks.com> wrote:
> If I understand your question, you are expecting all the containers to be
> allocated in one go? Or are you seeing your application hang because it
> asked for 10 containers but it only received a total of 9 even after
> repeated calls to the RM?
>
> There is no guarantee that you will be allocated all your requested
> containers in a single allocate() call. The application is expected to keep
> on making allocate calls to the RM at regular intervals and the RM will
> give it any newly allocated containers in the period since the last
> allocate call.
>
> The AMRMClient is a library that should make it easier for someone to
> write an application master. Yes - it is the recommended approach. Please
> file jiras if you see use-cases that are not addressed by it when
> implementing your AM.
>
> thanks
> -- Hitesh
>
> On Apr 3, 2013, at 8:05 AM, Krishna Kishore Bonagiri wrote:
>
> > Hi,
> >
> > I am running an application that had been written and working well with
> the hadoop-2.0.0-alpha but when I am running the same against 2.0.3-alpha,
> the getAllocatedContainers() method called on AMResponse is not returning
> all the containers allocated sometimes. For example, I request for 10
> containers and this method gives me only 9 containers sometimes, and when I
> looked at the log of Resource Manager, the 10th container is also
> allocated. It happens only sometimes randomly and works fine all other
> times. If I send one more request for the remaining container to RM, it
> could allocate that container. I am running only one application at a time,
> but 1000s of them one after another.
> >
> > Is this problem seen by anyone else? I have seen that there are
> significant number of changes in the way ApplicationMaster.java is written
> now for the Distributed Shell example. There is now AMRMClient instead of
> AMRMProtocol. There is method addContainerRequest() which is not there
> earlier, and there is ConatainerRequest class now which was ResourceRequest
> earlier, etc. Is it adivsable to re-write my application in a similar way?
> >
> > Thanks,
> > Kishore
> >
> >
>
>
Re: getAllocatedContainers() is not returning when ran against 2.0.3-alpha
Posted by Krishna Kishore Bonagiri <wr...@gmail.com>.
Hi Hitesh,
Thanks for the reply. I understand that it may not always allocate all
the requested containers in one go. After the actual request is sent, my
code repeatedly sends an empty container request and checks the responses
until all the requested containers are allocated.
But my main worry is, even though the RM's log is saying that all 10
requested containers are allocated, the getAllocatedContainers() method is
not returning me all of them, it returned only 9 surprisingly.
I shall file a jira for this, but please let me know soon if you have
any clue about this.
Thanks,
Kishore
On Wed, Apr 3, 2013 at 9:20 PM, Hitesh Shah <hi...@hortonworks.com> wrote:
> If I understand your question, you are expecting all the containers to be
> allocated in one go? Or are you seeing your application hang because it
> asked for 10 containers but it only received a total of 9 even after
> repeated calls to the RM?
>
> There is no guarantee that you will be allocated all your requested
> containers in a single allocate() call. The application is expected to keep
> on making allocate calls to the RM at regular intervals and the RM will
> give it any newly allocated containers in the period since the last
> allocate call.
>
> The AMRMClient is a library that should make it easier for someone to
> write an application master. Yes - it is the recommended approach. Please
> file jiras if you see use-cases that are not addressed by it when
> implementing your AM.
>
> thanks
> -- Hitesh
>
> On Apr 3, 2013, at 8:05 AM, Krishna Kishore Bonagiri wrote:
>
> > Hi,
> >
> > I am running an application that had been written and working well with
> the hadoop-2.0.0-alpha but when I am running the same against 2.0.3-alpha,
> the getAllocatedContainers() method called on AMResponse is not returning
> all the containers allocated sometimes. For example, I request for 10
> containers and this method gives me only 9 containers sometimes, and when I
> looked at the log of Resource Manager, the 10th container is also
> allocated. It happens only sometimes randomly and works fine all other
> times. If I send one more request for the remaining container to RM, it
> could allocate that container. I am running only one application at a time,
> but 1000s of them one after another.
> >
> > Is this problem seen by anyone else? I have seen that there are
> significant number of changes in the way ApplicationMaster.java is written
> now for the Distributed Shell example. There is now AMRMClient instead of
> AMRMProtocol. There is method addContainerRequest() which is not there
> earlier, and there is ConatainerRequest class now which was ResourceRequest
> earlier, etc. Is it adivsable to re-write my application in a similar way?
> >
> > Thanks,
> > Kishore
> >
> >
>
>
Re: getAllocatedContainers() is not returning when ran against 2.0.3-alpha
Posted by Hitesh Shah <hi...@hortonworks.com>.
If I understand your question, you are expecting all the containers to be allocated in one go? Or are you seeing your application hang because it asked for 10 containers but it only received a total of 9 even after repeated calls to the RM?
There is no guarantee that you will be allocated all your requested containers in a single allocate() call. The application is expected to keep on making allocate calls to the RM at regular intervals and the RM will give it any newly allocated containers in the period since the last allocate call.
The AMRMClient is a library that should make it easier for someone to write an application master. Yes - it is the recommended approach. Please file jiras if you see use-cases that are not addressed by it when implementing your AM.
thanks
-- Hitesh
On Apr 3, 2013, at 8:05 AM, Krishna Kishore Bonagiri wrote:
> Hi,
>
> I am running an application that had been written and working well with the hadoop-2.0.0-alpha but when I am running the same against 2.0.3-alpha, the getAllocatedContainers() method called on AMResponse is not returning all the containers allocated sometimes. For example, I request for 10 containers and this method gives me only 9 containers sometimes, and when I looked at the log of Resource Manager, the 10th container is also allocated. It happens only sometimes randomly and works fine all other times. If I send one more request for the remaining container to RM, it could allocate that container. I am running only one application at a time, but 1000s of them one after another.
>
> Is this problem seen by anyone else? I have seen that there are significant number of changes in the way ApplicationMaster.java is written now for the Distributed Shell example. There is now AMRMClient instead of AMRMProtocol. There is method addContainerRequest() which is not there earlier, and there is ConatainerRequest class now which was ResourceRequest earlier, etc. Is it adivsable to re-write my application in a similar way?
>
> Thanks,
> Kishore
>
>
Re: getAllocatedContainers() is not returning when ran against 2.0.3-alpha
Posted by Hitesh Shah <hi...@hortonworks.com>.
If I understand your question, you are expecting all the containers to be allocated in one go? Or are you seeing your application hang because it asked for 10 containers but it only received a total of 9 even after repeated calls to the RM?
There is no guarantee that you will be allocated all your requested containers in a single allocate() call. The application is expected to keep on making allocate calls to the RM at regular intervals and the RM will give it any newly allocated containers in the period since the last allocate call.
The AMRMClient is a library that should make it easier for someone to write an application master. Yes - it is the recommended approach. Please file jiras if you see use-cases that are not addressed by it when implementing your AM.
thanks
-- Hitesh
On Apr 3, 2013, at 8:05 AM, Krishna Kishore Bonagiri wrote:
> Hi,
>
> I am running an application that had been written and working well with the hadoop-2.0.0-alpha but when I am running the same against 2.0.3-alpha, the getAllocatedContainers() method called on AMResponse is not returning all the containers allocated sometimes. For example, I request for 10 containers and this method gives me only 9 containers sometimes, and when I looked at the log of Resource Manager, the 10th container is also allocated. It happens only sometimes randomly and works fine all other times. If I send one more request for the remaining container to RM, it could allocate that container. I am running only one application at a time, but 1000s of them one after another.
>
> Is this problem seen by anyone else? I have seen that there are significant number of changes in the way ApplicationMaster.java is written now for the Distributed Shell example. There is now AMRMClient instead of AMRMProtocol. There is method addContainerRequest() which is not there earlier, and there is ConatainerRequest class now which was ResourceRequest earlier, etc. Is it adivsable to re-write my application in a similar way?
>
> Thanks,
> Kishore
>
>
Re: getAllocatedContainers() is not returning when ran against 2.0.3-alpha
Posted by Hitesh Shah <hi...@hortonworks.com>.
If I understand your question, you are expecting all the containers to be allocated in one go? Or are you seeing your application hang because it asked for 10 containers but it only received a total of 9 even after repeated calls to the RM?
There is no guarantee that you will be allocated all your requested containers in a single allocate() call. The application is expected to keep on making allocate calls to the RM at regular intervals and the RM will give it any newly allocated containers in the period since the last allocate call.
The AMRMClient is a library that should make it easier for someone to write an application master. Yes - it is the recommended approach. Please file jiras if you see use-cases that are not addressed by it when implementing your AM.
thanks
-- Hitesh
On Apr 3, 2013, at 8:05 AM, Krishna Kishore Bonagiri wrote:
> Hi,
>
> I am running an application that had been written and working well with the hadoop-2.0.0-alpha but when I am running the same against 2.0.3-alpha, the getAllocatedContainers() method called on AMResponse is not returning all the containers allocated sometimes. For example, I request for 10 containers and this method gives me only 9 containers sometimes, and when I looked at the log of Resource Manager, the 10th container is also allocated. It happens only sometimes randomly and works fine all other times. If I send one more request for the remaining container to RM, it could allocate that container. I am running only one application at a time, but 1000s of them one after another.
>
> Is this problem seen by anyone else? I have seen that there are significant number of changes in the way ApplicationMaster.java is written now for the Distributed Shell example. There is now AMRMClient instead of AMRMProtocol. There is method addContainerRequest() which is not there earlier, and there is ConatainerRequest class now which was ResourceRequest earlier, etc. Is it adivsable to re-write my application in a similar way?
>
> Thanks,
> Kishore
>
>
Re: getAllocatedContainers() is not returning when ran against 2.0.3-alpha
Posted by Hitesh Shah <hi...@hortonworks.com>.
If I understand your question, you are expecting all the containers to be allocated in one go? Or are you seeing your application hang because it asked for 10 containers but it only received a total of 9 even after repeated calls to the RM?
There is no guarantee that you will be allocated all your requested containers in a single allocate() call. The application is expected to keep on making allocate calls to the RM at regular intervals and the RM will give it any newly allocated containers in the period since the last allocate call.
The AMRMClient is a library that should make it easier for someone to write an application master. Yes - it is the recommended approach. Please file jiras if you see use-cases that are not addressed by it when implementing your AM.
thanks
-- Hitesh
On Apr 3, 2013, at 8:05 AM, Krishna Kishore Bonagiri wrote:
> Hi,
>
> I am running an application that had been written and working well with the hadoop-2.0.0-alpha but when I am running the same against 2.0.3-alpha, the getAllocatedContainers() method called on AMResponse is not returning all the containers allocated sometimes. For example, I request for 10 containers and this method gives me only 9 containers sometimes, and when I looked at the log of Resource Manager, the 10th container is also allocated. It happens only sometimes randomly and works fine all other times. If I send one more request for the remaining container to RM, it could allocate that container. I am running only one application at a time, but 1000s of them one after another.
>
> Is this problem seen by anyone else? I have seen that there are significant number of changes in the way ApplicationMaster.java is written now for the Distributed Shell example. There is now AMRMClient instead of AMRMProtocol. There is method addContainerRequest() which is not there earlier, and there is ConatainerRequest class now which was ResourceRequest earlier, etc. Is it adivsable to re-write my application in a similar way?
>
> Thanks,
> Kishore
>
>