You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by Hyunsik Choi <hy...@apache.org> on 2011/08/10 06:28:05 UTC
Adaptive resource assignment is possible in MRv2?
Hi guys,
MRv2 looks very promising for various applications. We have considered
an application based on Yarn, but this app is required the adaptive
resource assignment. In other words, this app may need additional
resources while this app is running. Is it possible?
After I look at some code, especially YARNRunner, ResourceMgrDelegate,
and ClientRMProtocol. In these classes, an application seems to be
submitted with necessary resource capacity description. So, it is
impossible?
Best regards,
--
Hyunsik Choi
Re: Adaptive resource assignment is possible in MRv2?
Posted by Hyunsik Choi <hy...@apache.org>.
I'm sorry for my poor English. I correct some English errors, and I post as
follows:
----
MRv2 looks very promising for various applications. We have considered
an application based on Yarn, but this app requires the adaptive
resource assignment. In other words, this app may need additional
resources while this app is running. Is it possible?
I looked at some code, especially YARNRunner, ResourceMgrDelegate,
and ClientRMProtocol. In these classes, an application seems to be
submitted with necessary resource capacity description.
So, adaptive resource assignment is impossible?
On Wed, Aug 10, 2011 at 1:28 PM, Hyunsik Choi <hy...@apache.org> wrote:
> Hi guys,
>
> MRv2 looks very promising for various applications. We have considered
> an application based on Yarn, but this app is required the adaptive
> resource assignment. In other words, this app may need additional
> resources while this app is running. Is it possible?
>
> After I look at some code, especially YARNRunner, ResourceMgrDelegate,
> and ClientRMProtocol. In these classes, an application seems to be
> submitted with necessary resource capacity description. So, it is
> impossible?
>
> Best regards,
> --
> Hyunsik Choi
>
Re: Adaptive resource assignment is possible in MRv2?
Posted by Hyunsik Choi <hy...@gmail.com>.
Thank you for helpful reply :)
Cheers,
--
Hyunsik Choi
On Wed, Aug 10, 2011 at 2:23 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
> On Aug 9, 2011, at 9:28 PM, Hyunsik Choi wrote:
>
> > Hi guys,
> >
> > MRv2 looks very promising for various applications. We have considered
> > an application based on Yarn, but this app is required the adaptive
> > resource assignment. In other words, this app may need additional
> > resources while this app is running. Is it possible?
> >
>
> Absolutely.
>
> > After I look at some code, especially YARNRunner, ResourceMgrDelegate,
> > and ClientRMProtocol. In these classes, an application seems to be
> > submitted with necessary resource capacity description. So, it is
> > impossible?
>
> Typically, MR applications present resource requirements for maps upfront
> (after split calculation), but submit reduce resource requirements later.
>
> So, yes - applications can change their resource requests at any point
> during their lifecycle.
>
> hth,
> Arun
>
>
Re: Adaptive resource assignment is possible in MRv2?
Posted by Arun C Murthy <ac...@hortonworks.com>.
On Aug 9, 2011, at 9:28 PM, Hyunsik Choi wrote:
> Hi guys,
>
> MRv2 looks very promising for various applications. We have considered
> an application based on Yarn, but this app is required the adaptive
> resource assignment. In other words, this app may need additional
> resources while this app is running. Is it possible?
>
Absolutely.
> After I look at some code, especially YARNRunner, ResourceMgrDelegate,
> and ClientRMProtocol. In these classes, an application seems to be
> submitted with necessary resource capacity description. So, it is
> impossible?
Typically, MR applications present resource requirements for maps upfront (after split calculation), but submit reduce resource requirements later.
So, yes - applications can change their resource requests at any point during their lifecycle.
hth,
Arun