You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Krishna Kishore Bonagiri <wr...@gmail.com> on 2014/04/08 16:53:42 UTC
Reuse of YARN container
Hi,
Does this JIRA issue mean that we can't currently reuse a container for
running/launching two different processes one after another?
https://issues.apache.org/jira/browse/YARN-373
If that is true, are there any plans for making that possible?
Thanks,
Kishore
Re: Reuse of YARN container
Posted by Tsuyoshi OZAWA <oz...@gmail.com>.
Hi,
As Oleg mentioned, container-reuse is done at ApplicationMaster level
currently. Tez's ApplicationMaster is one of them.
Thanks,
- Tsuyoshi
On Thu, Apr 24, 2014 at 2:03 AM, Oleg Zhurakousky
<ol...@gmail.com> wrote:
> While YARN-373 addresses a bit of a different problem the use case of reuse
> of existing resources could be looked at few different ways. As I understand
> YARN-373 talks about reusing resources from an old container to spun off a
> new one. Another way of looking at the problem is to actually reuse a
> container itself as long as you can communicate with it.
> I've been experimenting with it recently and while I am not sure if the
> approach is valid or solves your use case, take a look here
> https://github.com/olegz/yaya/wiki/CoreFeatures#java-based-long-running-and-interact-able-application-containers.
> At least you'll get some ideas of what could be done.
>
> Cheers
> Oleg
>
>
> On Tue, Apr 8, 2014 at 10:53 AM, Krishna Kishore Bonagiri
> <wr...@gmail.com> wrote:
>>
>> Hi,
>>
>> Does this JIRA issue mean that we can't currently reuse a container for
>> running/launching two different processes one after another?
>>
>> https://issues.apache.org/jira/browse/YARN-373
>>
>>
>> If that is true, are there any plans for making that possible?
>>
>> Thanks,
>> Kishore
>
>
--
- Tsuyoshi
Re: Reuse of YARN container
Posted by Tsuyoshi OZAWA <oz...@gmail.com>.
Hi,
As Oleg mentioned, container-reuse is done at ApplicationMaster level
currently. Tez's ApplicationMaster is one of them.
Thanks,
- Tsuyoshi
On Thu, Apr 24, 2014 at 2:03 AM, Oleg Zhurakousky
<ol...@gmail.com> wrote:
> While YARN-373 addresses a bit of a different problem the use case of reuse
> of existing resources could be looked at few different ways. As I understand
> YARN-373 talks about reusing resources from an old container to spun off a
> new one. Another way of looking at the problem is to actually reuse a
> container itself as long as you can communicate with it.
> I've been experimenting with it recently and while I am not sure if the
> approach is valid or solves your use case, take a look here
> https://github.com/olegz/yaya/wiki/CoreFeatures#java-based-long-running-and-interact-able-application-containers.
> At least you'll get some ideas of what could be done.
>
> Cheers
> Oleg
>
>
> On Tue, Apr 8, 2014 at 10:53 AM, Krishna Kishore Bonagiri
> <wr...@gmail.com> wrote:
>>
>> Hi,
>>
>> Does this JIRA issue mean that we can't currently reuse a container for
>> running/launching two different processes one after another?
>>
>> https://issues.apache.org/jira/browse/YARN-373
>>
>>
>> If that is true, are there any plans for making that possible?
>>
>> Thanks,
>> Kishore
>
>
--
- Tsuyoshi
Re: Reuse of YARN container
Posted by Tsuyoshi OZAWA <oz...@gmail.com>.
Hi,
As Oleg mentioned, container-reuse is done at ApplicationMaster level
currently. Tez's ApplicationMaster is one of them.
Thanks,
- Tsuyoshi
On Thu, Apr 24, 2014 at 2:03 AM, Oleg Zhurakousky
<ol...@gmail.com> wrote:
> While YARN-373 addresses a bit of a different problem the use case of reuse
> of existing resources could be looked at few different ways. As I understand
> YARN-373 talks about reusing resources from an old container to spun off a
> new one. Another way of looking at the problem is to actually reuse a
> container itself as long as you can communicate with it.
> I've been experimenting with it recently and while I am not sure if the
> approach is valid or solves your use case, take a look here
> https://github.com/olegz/yaya/wiki/CoreFeatures#java-based-long-running-and-interact-able-application-containers.
> At least you'll get some ideas of what could be done.
>
> Cheers
> Oleg
>
>
> On Tue, Apr 8, 2014 at 10:53 AM, Krishna Kishore Bonagiri
> <wr...@gmail.com> wrote:
>>
>> Hi,
>>
>> Does this JIRA issue mean that we can't currently reuse a container for
>> running/launching two different processes one after another?
>>
>> https://issues.apache.org/jira/browse/YARN-373
>>
>>
>> If that is true, are there any plans for making that possible?
>>
>> Thanks,
>> Kishore
>
>
--
- Tsuyoshi
Re: Reuse of YARN container
Posted by Tsuyoshi OZAWA <oz...@gmail.com>.
Hi,
As Oleg mentioned, container-reuse is done at ApplicationMaster level
currently. Tez's ApplicationMaster is one of them.
Thanks,
- Tsuyoshi
On Thu, Apr 24, 2014 at 2:03 AM, Oleg Zhurakousky
<ol...@gmail.com> wrote:
> While YARN-373 addresses a bit of a different problem the use case of reuse
> of existing resources could be looked at few different ways. As I understand
> YARN-373 talks about reusing resources from an old container to spun off a
> new one. Another way of looking at the problem is to actually reuse a
> container itself as long as you can communicate with it.
> I've been experimenting with it recently and while I am not sure if the
> approach is valid or solves your use case, take a look here
> https://github.com/olegz/yaya/wiki/CoreFeatures#java-based-long-running-and-interact-able-application-containers.
> At least you'll get some ideas of what could be done.
>
> Cheers
> Oleg
>
>
> On Tue, Apr 8, 2014 at 10:53 AM, Krishna Kishore Bonagiri
> <wr...@gmail.com> wrote:
>>
>> Hi,
>>
>> Does this JIRA issue mean that we can't currently reuse a container for
>> running/launching two different processes one after another?
>>
>> https://issues.apache.org/jira/browse/YARN-373
>>
>>
>> If that is true, are there any plans for making that possible?
>>
>> Thanks,
>> Kishore
>
>
--
- Tsuyoshi
Re: Reuse of YARN container
Posted by Oleg Zhurakousky <ol...@gmail.com>.
While YARN-373 addresses a bit of a different problem the use case of reuse
of existing resources could be looked at few different ways. As I
understand YARN-373 talks about reusing resources from an old container to
spun off a new one. Another way of looking at the problem is to actually
reuse a container itself as long as you can communicate with it.
I've been experimenting with it recently and while I am not sure if the
approach is valid or solves your use case, take a look here
https://github.com/olegz/yaya/wiki/CoreFeatures#java-based-long-running-and-interact-able-application-containers.
At least you'll get some ideas of what could be done.
Cheers
Oleg
On Tue, Apr 8, 2014 at 10:53 AM, Krishna Kishore Bonagiri <
write2kishore@gmail.com> wrote:
> Hi,
>
> Does this JIRA issue mean that we can't currently reuse a container for
> running/launching two different processes one after another?
>
> https://issues.apache.org/jira/browse/YARN-373
>
>
> If that is true, are there any plans for making that possible?
>
> Thanks,
> Kishore
>
Re: Reuse of YARN container
Posted by Oleg Zhurakousky <ol...@gmail.com>.
While YARN-373 addresses a bit of a different problem the use case of reuse
of existing resources could be looked at few different ways. As I
understand YARN-373 talks about reusing resources from an old container to
spun off a new one. Another way of looking at the problem is to actually
reuse a container itself as long as you can communicate with it.
I've been experimenting with it recently and while I am not sure if the
approach is valid or solves your use case, take a look here
https://github.com/olegz/yaya/wiki/CoreFeatures#java-based-long-running-and-interact-able-application-containers.
At least you'll get some ideas of what could be done.
Cheers
Oleg
On Tue, Apr 8, 2014 at 10:53 AM, Krishna Kishore Bonagiri <
write2kishore@gmail.com> wrote:
> Hi,
>
> Does this JIRA issue mean that we can't currently reuse a container for
> running/launching two different processes one after another?
>
> https://issues.apache.org/jira/browse/YARN-373
>
>
> If that is true, are there any plans for making that possible?
>
> Thanks,
> Kishore
>
Re: Reuse of YARN container
Posted by Oleg Zhurakousky <ol...@gmail.com>.
While YARN-373 addresses a bit of a different problem the use case of reuse
of existing resources could be looked at few different ways. As I
understand YARN-373 talks about reusing resources from an old container to
spun off a new one. Another way of looking at the problem is to actually
reuse a container itself as long as you can communicate with it.
I've been experimenting with it recently and while I am not sure if the
approach is valid or solves your use case, take a look here
https://github.com/olegz/yaya/wiki/CoreFeatures#java-based-long-running-and-interact-able-application-containers.
At least you'll get some ideas of what could be done.
Cheers
Oleg
On Tue, Apr 8, 2014 at 10:53 AM, Krishna Kishore Bonagiri <
write2kishore@gmail.com> wrote:
> Hi,
>
> Does this JIRA issue mean that we can't currently reuse a container for
> running/launching two different processes one after another?
>
> https://issues.apache.org/jira/browse/YARN-373
>
>
> If that is true, are there any plans for making that possible?
>
> Thanks,
> Kishore
>
Re: Reuse of YARN container
Posted by Oleg Zhurakousky <ol...@gmail.com>.
While YARN-373 addresses a bit of a different problem the use case of reuse
of existing resources could be looked at few different ways. As I
understand YARN-373 talks about reusing resources from an old container to
spun off a new one. Another way of looking at the problem is to actually
reuse a container itself as long as you can communicate with it.
I've been experimenting with it recently and while I am not sure if the
approach is valid or solves your use case, take a look here
https://github.com/olegz/yaya/wiki/CoreFeatures#java-based-long-running-and-interact-able-application-containers.
At least you'll get some ideas of what could be done.
Cheers
Oleg
On Tue, Apr 8, 2014 at 10:53 AM, Krishna Kishore Bonagiri <
write2kishore@gmail.com> wrote:
> Hi,
>
> Does this JIRA issue mean that we can't currently reuse a container for
> running/launching two different processes one after another?
>
> https://issues.apache.org/jira/browse/YARN-373
>
>
> If that is true, are there any plans for making that possible?
>
> Thanks,
> Kishore
>