You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Zoltán Zvara <zo...@gmail.com> on 2016/03/29 18:45:42 UTC
YARN re-locate container
Dear Hadoop Community,
Is there any feature available, or on the road map to support the
relocation of containers? (Simply restart the container on another machine.)
Thanks,
Zoltán
Re: YARN re-locate container
Posted by Musty Rehmani <mu...@yahoo.com.INVALID>.
You can check memory allocation at yarn-site. xml
Sent from Yahoo Mail on Android
On Thu, Mar 31, 2016 at 2:18 PM, Zoltán Zvara<zo...@gmail.com> wrote: Yes, I'm aware of that, but here we have very-long running containers. Nodes are from 100s to 1000s, it depends on the site. Memory and heap sizes are not known.
On Thu, Mar 31, 2016 at 1:56 PM Musty Rehmani <mu...@yahoo.com> wrote:
Containers are temporary. They shut down after job is finished. How many nodes in your cluster and what is the memory and heap size
Sent from Yahoo Mail on Android
On Thu, Mar 31, 2016 at 4:44 AM, Zoltán Zvara
<zo...@gmail.com> wrote: The intent to move a container is to improve service-to-service locality, co-locate services to reduce bottlenecks introduced by network. The idea that a certain parallel process should be moved comes from an external service.
Speculation might not be effective here as I see, since the intent to move a container can be triggered hourly, daily, weekly or monthly - containers contain long running services, data pipelines. Or am I on the wrong line here? Can I simulate something like this with the current speculation?
Thanks for help!
On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com> wrote:
I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
Thanks,-Eric
From: Zoltán Zvara <zo...@gmail.com>
To: Vinod Kumar Vavilapalli <vi...@apache.org>
Cc: user@hadoop.apache.org
Sent: Wednesday, March 30, 2016 9:10 AM
Subject: Re: YARN re-locate container
How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
Re: YARN re-locate container
Posted by Musty Rehmani <mu...@yahoo.com.INVALID>.
You can check memory allocation at yarn-site. xml
Sent from Yahoo Mail on Android
On Thu, Mar 31, 2016 at 2:18 PM, Zoltán Zvara<zo...@gmail.com> wrote: Yes, I'm aware of that, but here we have very-long running containers. Nodes are from 100s to 1000s, it depends on the site. Memory and heap sizes are not known.
On Thu, Mar 31, 2016 at 1:56 PM Musty Rehmani <mu...@yahoo.com> wrote:
Containers are temporary. They shut down after job is finished. How many nodes in your cluster and what is the memory and heap size
Sent from Yahoo Mail on Android
On Thu, Mar 31, 2016 at 4:44 AM, Zoltán Zvara
<zo...@gmail.com> wrote: The intent to move a container is to improve service-to-service locality, co-locate services to reduce bottlenecks introduced by network. The idea that a certain parallel process should be moved comes from an external service.
Speculation might not be effective here as I see, since the intent to move a container can be triggered hourly, daily, weekly or monthly - containers contain long running services, data pipelines. Or am I on the wrong line here? Can I simulate something like this with the current speculation?
Thanks for help!
On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com> wrote:
I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
Thanks,-Eric
From: Zoltán Zvara <zo...@gmail.com>
To: Vinod Kumar Vavilapalli <vi...@apache.org>
Cc: user@hadoop.apache.org
Sent: Wednesday, March 30, 2016 9:10 AM
Subject: Re: YARN re-locate container
How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
Re: YARN re-locate container
Posted by Musty Rehmani <mu...@yahoo.com.INVALID>.
You can check memory allocation at yarn-site. xml
Sent from Yahoo Mail on Android
On Thu, Mar 31, 2016 at 2:18 PM, Zoltán Zvara<zo...@gmail.com> wrote: Yes, I'm aware of that, but here we have very-long running containers. Nodes are from 100s to 1000s, it depends on the site. Memory and heap sizes are not known.
On Thu, Mar 31, 2016 at 1:56 PM Musty Rehmani <mu...@yahoo.com> wrote:
Containers are temporary. They shut down after job is finished. How many nodes in your cluster and what is the memory and heap size
Sent from Yahoo Mail on Android
On Thu, Mar 31, 2016 at 4:44 AM, Zoltán Zvara
<zo...@gmail.com> wrote: The intent to move a container is to improve service-to-service locality, co-locate services to reduce bottlenecks introduced by network. The idea that a certain parallel process should be moved comes from an external service.
Speculation might not be effective here as I see, since the intent to move a container can be triggered hourly, daily, weekly or monthly - containers contain long running services, data pipelines. Or am I on the wrong line here? Can I simulate something like this with the current speculation?
Thanks for help!
On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com> wrote:
I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
Thanks,-Eric
From: Zoltán Zvara <zo...@gmail.com>
To: Vinod Kumar Vavilapalli <vi...@apache.org>
Cc: user@hadoop.apache.org
Sent: Wednesday, March 30, 2016 9:10 AM
Subject: Re: YARN re-locate container
How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
Re: YARN re-locate container
Posted by Musty Rehmani <mu...@yahoo.com.INVALID>.
You can check memory allocation at yarn-site. xml
Sent from Yahoo Mail on Android
On Thu, Mar 31, 2016 at 2:18 PM, Zoltán Zvara<zo...@gmail.com> wrote: Yes, I'm aware of that, but here we have very-long running containers. Nodes are from 100s to 1000s, it depends on the site. Memory and heap sizes are not known.
On Thu, Mar 31, 2016 at 1:56 PM Musty Rehmani <mu...@yahoo.com> wrote:
Containers are temporary. They shut down after job is finished. How many nodes in your cluster and what is the memory and heap size
Sent from Yahoo Mail on Android
On Thu, Mar 31, 2016 at 4:44 AM, Zoltán Zvara
<zo...@gmail.com> wrote: The intent to move a container is to improve service-to-service locality, co-locate services to reduce bottlenecks introduced by network. The idea that a certain parallel process should be moved comes from an external service.
Speculation might not be effective here as I see, since the intent to move a container can be triggered hourly, daily, weekly or monthly - containers contain long running services, data pipelines. Or am I on the wrong line here? Can I simulate something like this with the current speculation?
Thanks for help!
On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com> wrote:
I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
Thanks,-Eric
From: Zoltán Zvara <zo...@gmail.com>
To: Vinod Kumar Vavilapalli <vi...@apache.org>
Cc: user@hadoop.apache.org
Sent: Wednesday, March 30, 2016 9:10 AM
Subject: Re: YARN re-locate container
How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
Re: YARN re-locate container
Posted by Zoltán Zvara <zo...@gmail.com>.
Yes, I'm aware of that, but here we have very-long running containers.
Nodes are from 100s to 1000s, it depends on the site. Memory and heap sizes
are not known.
On Thu, Mar 31, 2016 at 1:56 PM Musty Rehmani <mu...@yahoo.com>
wrote:
> Containers are temporary. They shut down after job is finished. How many
> nodes in your cluster and what is the memory and heap size
>
>
>
> Sent from Yahoo Mail on Android
> <https://overview.mail.yahoo.com/mobile/?.src=Android>
>
> On Thu, Mar 31, 2016 at 4:44 AM, Zoltán Zvara
>
> <zo...@gmail.com> wrote:
> The intent to move a container is to improve service-to-service locality,
> co-locate services to reduce bottlenecks introduced by network. The idea
> that a certain parallel process should be moved comes from an external
> service.
>
> Speculation might not be effective here as I see, since the intent to move
> a container can be triggered hourly, daily, weekly or monthly - containers
> contain long running services, data pipelines. Or am I on the wrong line
> here? Can I simulate something like this with the current speculation?
>
> Thanks for help!
>
> On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com>
> wrote:
>
>> I think it would help if I knew what the criteria is for wanting to move
>> the container. In other words, was the container started on an undesirable
>> node in the first place? Or, did the node become undesirable after the
>> container started.
>>
>> Speculation could be considered a "move" operation for containers. If a
>> container isn't finishing fast enough, the default speculator will start
>> another container on a different node. Would it be possible to create a
>> specialized speculator that understood your criteria for needing to move
>> containers? If so, it could be done automatically / programatically.
>>
>> Thanks,
>> -Eric
>>
>>
>> ------------------------------
>> *From:* Zoltán Zvara <zo...@gmail.com>
>> *To:* Vinod Kumar Vavilapalli <vi...@apache.org>
>> *Cc:* user@hadoop.apache.org
>> *Sent:* Wednesday, March 30, 2016 9:10 AM
>> *Subject:* Re: YARN re-locate container
>>
>> How is this achieved? As far as I see it now, after stopping a container,
>> the AM must reallocate the same container with the same resource vector but
>> with locality preferences pointed to the new, target node. After the new
>> leash has been acquired, then the AM can take it to the new node and
>> initiate a `startContainers` message.
>> Our use-case with Ericsson would require a more simple API, where (for
>> example) a `moveContainer` call from the AM would ask the RM or NM to move
>> a container from one node to another (or to any of the specified set of
>> preferred nodes). Move would simply kill the container and restart it on
>> another node at any given time whenever it is possible - I feel questions
>> around scheduling: how container moves should be handled? Probably not like
>> simple allocations.
>>
>> Am I understanding the architecture correctly here?
>>
>> On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <
>> vinodkv@apache.org> wrote:
>>
>> Containers can be restarted on other machines already today - YARN just
>> leaves it up to the applications to do so.
>>
>> Are you looking for anything more specifically?
>>
>> +Vinod
>>
>> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com>
>> wrote:
>> >
>> > Dear Hadoop Community,
>> >
>> > Is there any feature available, or on the road map to support the
>> relocation of containers? (Simply restart the container on another machine.)
>> >
>> > Thanks,
>> > Zoltán
>>
>>
>>
>>
Re: YARN re-locate container
Posted by Zoltán Zvara <zo...@gmail.com>.
Yes, I'm aware of that, but here we have very-long running containers.
Nodes are from 100s to 1000s, it depends on the site. Memory and heap sizes
are not known.
On Thu, Mar 31, 2016 at 1:56 PM Musty Rehmani <mu...@yahoo.com>
wrote:
> Containers are temporary. They shut down after job is finished. How many
> nodes in your cluster and what is the memory and heap size
>
>
>
> Sent from Yahoo Mail on Android
> <https://overview.mail.yahoo.com/mobile/?.src=Android>
>
> On Thu, Mar 31, 2016 at 4:44 AM, Zoltán Zvara
>
> <zo...@gmail.com> wrote:
> The intent to move a container is to improve service-to-service locality,
> co-locate services to reduce bottlenecks introduced by network. The idea
> that a certain parallel process should be moved comes from an external
> service.
>
> Speculation might not be effective here as I see, since the intent to move
> a container can be triggered hourly, daily, weekly or monthly - containers
> contain long running services, data pipelines. Or am I on the wrong line
> here? Can I simulate something like this with the current speculation?
>
> Thanks for help!
>
> On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com>
> wrote:
>
>> I think it would help if I knew what the criteria is for wanting to move
>> the container. In other words, was the container started on an undesirable
>> node in the first place? Or, did the node become undesirable after the
>> container started.
>>
>> Speculation could be considered a "move" operation for containers. If a
>> container isn't finishing fast enough, the default speculator will start
>> another container on a different node. Would it be possible to create a
>> specialized speculator that understood your criteria for needing to move
>> containers? If so, it could be done automatically / programatically.
>>
>> Thanks,
>> -Eric
>>
>>
>> ------------------------------
>> *From:* Zoltán Zvara <zo...@gmail.com>
>> *To:* Vinod Kumar Vavilapalli <vi...@apache.org>
>> *Cc:* user@hadoop.apache.org
>> *Sent:* Wednesday, March 30, 2016 9:10 AM
>> *Subject:* Re: YARN re-locate container
>>
>> How is this achieved? As far as I see it now, after stopping a container,
>> the AM must reallocate the same container with the same resource vector but
>> with locality preferences pointed to the new, target node. After the new
>> leash has been acquired, then the AM can take it to the new node and
>> initiate a `startContainers` message.
>> Our use-case with Ericsson would require a more simple API, where (for
>> example) a `moveContainer` call from the AM would ask the RM or NM to move
>> a container from one node to another (or to any of the specified set of
>> preferred nodes). Move would simply kill the container and restart it on
>> another node at any given time whenever it is possible - I feel questions
>> around scheduling: how container moves should be handled? Probably not like
>> simple allocations.
>>
>> Am I understanding the architecture correctly here?
>>
>> On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <
>> vinodkv@apache.org> wrote:
>>
>> Containers can be restarted on other machines already today - YARN just
>> leaves it up to the applications to do so.
>>
>> Are you looking for anything more specifically?
>>
>> +Vinod
>>
>> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com>
>> wrote:
>> >
>> > Dear Hadoop Community,
>> >
>> > Is there any feature available, or on the road map to support the
>> relocation of containers? (Simply restart the container on another machine.)
>> >
>> > Thanks,
>> > Zoltán
>>
>>
>>
>>
Re: YARN re-locate container
Posted by Zoltán Zvara <zo...@gmail.com>.
Yes, I'm aware of that, but here we have very-long running containers.
Nodes are from 100s to 1000s, it depends on the site. Memory and heap sizes
are not known.
On Thu, Mar 31, 2016 at 1:56 PM Musty Rehmani <mu...@yahoo.com>
wrote:
> Containers are temporary. They shut down after job is finished. How many
> nodes in your cluster and what is the memory and heap size
>
>
>
> Sent from Yahoo Mail on Android
> <https://overview.mail.yahoo.com/mobile/?.src=Android>
>
> On Thu, Mar 31, 2016 at 4:44 AM, Zoltán Zvara
>
> <zo...@gmail.com> wrote:
> The intent to move a container is to improve service-to-service locality,
> co-locate services to reduce bottlenecks introduced by network. The idea
> that a certain parallel process should be moved comes from an external
> service.
>
> Speculation might not be effective here as I see, since the intent to move
> a container can be triggered hourly, daily, weekly or monthly - containers
> contain long running services, data pipelines. Or am I on the wrong line
> here? Can I simulate something like this with the current speculation?
>
> Thanks for help!
>
> On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com>
> wrote:
>
>> I think it would help if I knew what the criteria is for wanting to move
>> the container. In other words, was the container started on an undesirable
>> node in the first place? Or, did the node become undesirable after the
>> container started.
>>
>> Speculation could be considered a "move" operation for containers. If a
>> container isn't finishing fast enough, the default speculator will start
>> another container on a different node. Would it be possible to create a
>> specialized speculator that understood your criteria for needing to move
>> containers? If so, it could be done automatically / programatically.
>>
>> Thanks,
>> -Eric
>>
>>
>> ------------------------------
>> *From:* Zoltán Zvara <zo...@gmail.com>
>> *To:* Vinod Kumar Vavilapalli <vi...@apache.org>
>> *Cc:* user@hadoop.apache.org
>> *Sent:* Wednesday, March 30, 2016 9:10 AM
>> *Subject:* Re: YARN re-locate container
>>
>> How is this achieved? As far as I see it now, after stopping a container,
>> the AM must reallocate the same container with the same resource vector but
>> with locality preferences pointed to the new, target node. After the new
>> leash has been acquired, then the AM can take it to the new node and
>> initiate a `startContainers` message.
>> Our use-case with Ericsson would require a more simple API, where (for
>> example) a `moveContainer` call from the AM would ask the RM or NM to move
>> a container from one node to another (or to any of the specified set of
>> preferred nodes). Move would simply kill the container and restart it on
>> another node at any given time whenever it is possible - I feel questions
>> around scheduling: how container moves should be handled? Probably not like
>> simple allocations.
>>
>> Am I understanding the architecture correctly here?
>>
>> On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <
>> vinodkv@apache.org> wrote:
>>
>> Containers can be restarted on other machines already today - YARN just
>> leaves it up to the applications to do so.
>>
>> Are you looking for anything more specifically?
>>
>> +Vinod
>>
>> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com>
>> wrote:
>> >
>> > Dear Hadoop Community,
>> >
>> > Is there any feature available, or on the road map to support the
>> relocation of containers? (Simply restart the container on another machine.)
>> >
>> > Thanks,
>> > Zoltán
>>
>>
>>
>>
Re: YARN re-locate container
Posted by Zoltán Zvara <zo...@gmail.com>.
Yes, I'm aware of that, but here we have very-long running containers.
Nodes are from 100s to 1000s, it depends on the site. Memory and heap sizes
are not known.
On Thu, Mar 31, 2016 at 1:56 PM Musty Rehmani <mu...@yahoo.com>
wrote:
> Containers are temporary. They shut down after job is finished. How many
> nodes in your cluster and what is the memory and heap size
>
>
>
> Sent from Yahoo Mail on Android
> <https://overview.mail.yahoo.com/mobile/?.src=Android>
>
> On Thu, Mar 31, 2016 at 4:44 AM, Zoltán Zvara
>
> <zo...@gmail.com> wrote:
> The intent to move a container is to improve service-to-service locality,
> co-locate services to reduce bottlenecks introduced by network. The idea
> that a certain parallel process should be moved comes from an external
> service.
>
> Speculation might not be effective here as I see, since the intent to move
> a container can be triggered hourly, daily, weekly or monthly - containers
> contain long running services, data pipelines. Or am I on the wrong line
> here? Can I simulate something like this with the current speculation?
>
> Thanks for help!
>
> On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com>
> wrote:
>
>> I think it would help if I knew what the criteria is for wanting to move
>> the container. In other words, was the container started on an undesirable
>> node in the first place? Or, did the node become undesirable after the
>> container started.
>>
>> Speculation could be considered a "move" operation for containers. If a
>> container isn't finishing fast enough, the default speculator will start
>> another container on a different node. Would it be possible to create a
>> specialized speculator that understood your criteria for needing to move
>> containers? If so, it could be done automatically / programatically.
>>
>> Thanks,
>> -Eric
>>
>>
>> ------------------------------
>> *From:* Zoltán Zvara <zo...@gmail.com>
>> *To:* Vinod Kumar Vavilapalli <vi...@apache.org>
>> *Cc:* user@hadoop.apache.org
>> *Sent:* Wednesday, March 30, 2016 9:10 AM
>> *Subject:* Re: YARN re-locate container
>>
>> How is this achieved? As far as I see it now, after stopping a container,
>> the AM must reallocate the same container with the same resource vector but
>> with locality preferences pointed to the new, target node. After the new
>> leash has been acquired, then the AM can take it to the new node and
>> initiate a `startContainers` message.
>> Our use-case with Ericsson would require a more simple API, where (for
>> example) a `moveContainer` call from the AM would ask the RM or NM to move
>> a container from one node to another (or to any of the specified set of
>> preferred nodes). Move would simply kill the container and restart it on
>> another node at any given time whenever it is possible - I feel questions
>> around scheduling: how container moves should be handled? Probably not like
>> simple allocations.
>>
>> Am I understanding the architecture correctly here?
>>
>> On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <
>> vinodkv@apache.org> wrote:
>>
>> Containers can be restarted on other machines already today - YARN just
>> leaves it up to the applications to do so.
>>
>> Are you looking for anything more specifically?
>>
>> +Vinod
>>
>> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com>
>> wrote:
>> >
>> > Dear Hadoop Community,
>> >
>> > Is there any feature available, or on the road map to support the
>> relocation of containers? (Simply restart the container on another machine.)
>> >
>> > Thanks,
>> > Zoltán
>>
>>
>>
>>
Re: YARN re-locate container
Posted by Musty Rehmani <mu...@yahoo.com.INVALID>.
Containers are temporary. They shut down after job is finished. How many nodes in your cluster and what is the memory and heap size
Sent from Yahoo Mail on Android
On Thu, Mar 31, 2016 at 4:44 AM, Zoltán Zvara<zo...@gmail.com> wrote: The intent to move a container is to improve service-to-service locality, co-locate services to reduce bottlenecks introduced by network. The idea that a certain parallel process should be moved comes from an external service.
Speculation might not be effective here as I see, since the intent to move a container can be triggered hourly, daily, weekly or monthly - containers contain long running services, data pipelines. Or am I on the wrong line here? Can I simulate something like this with the current speculation?
Thanks for help!
On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com> wrote:
I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
Thanks,-Eric
From: Zoltán Zvara <zo...@gmail.com>
To: Vinod Kumar Vavilapalli <vi...@apache.org>
Cc: user@hadoop.apache.org
Sent: Wednesday, March 30, 2016 9:10 AM
Subject: Re: YARN re-locate container
How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
Re: YARN re-locate container
Posted by Musty Rehmani <mu...@yahoo.com.INVALID>.
Containers are temporary. They shut down after job is finished. How many nodes in your cluster and what is the memory and heap size
Sent from Yahoo Mail on Android
On Thu, Mar 31, 2016 at 4:44 AM, Zoltán Zvara<zo...@gmail.com> wrote: The intent to move a container is to improve service-to-service locality, co-locate services to reduce bottlenecks introduced by network. The idea that a certain parallel process should be moved comes from an external service.
Speculation might not be effective here as I see, since the intent to move a container can be triggered hourly, daily, weekly or monthly - containers contain long running services, data pipelines. Or am I on the wrong line here? Can I simulate something like this with the current speculation?
Thanks for help!
On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com> wrote:
I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
Thanks,-Eric
From: Zoltán Zvara <zo...@gmail.com>
To: Vinod Kumar Vavilapalli <vi...@apache.org>
Cc: user@hadoop.apache.org
Sent: Wednesday, March 30, 2016 9:10 AM
Subject: Re: YARN re-locate container
How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
Re: YARN re-locate container
Posted by Musty Rehmani <mu...@yahoo.com.INVALID>.
Containers are temporary. They shut down after job is finished. How many nodes in your cluster and what is the memory and heap size
Sent from Yahoo Mail on Android
On Thu, Mar 31, 2016 at 4:44 AM, Zoltán Zvara<zo...@gmail.com> wrote: The intent to move a container is to improve service-to-service locality, co-locate services to reduce bottlenecks introduced by network. The idea that a certain parallel process should be moved comes from an external service.
Speculation might not be effective here as I see, since the intent to move a container can be triggered hourly, daily, weekly or monthly - containers contain long running services, data pipelines. Or am I on the wrong line here? Can I simulate something like this with the current speculation?
Thanks for help!
On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com> wrote:
I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
Thanks,-Eric
From: Zoltán Zvara <zo...@gmail.com>
To: Vinod Kumar Vavilapalli <vi...@apache.org>
Cc: user@hadoop.apache.org
Sent: Wednesday, March 30, 2016 9:10 AM
Subject: Re: YARN re-locate container
How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
Re: YARN re-locate container
Posted by Musty Rehmani <mu...@yahoo.com.INVALID>.
Containers are temporary. They shut down after job is finished. How many nodes in your cluster and what is the memory and heap size
Sent from Yahoo Mail on Android
On Thu, Mar 31, 2016 at 4:44 AM, Zoltán Zvara<zo...@gmail.com> wrote: The intent to move a container is to improve service-to-service locality, co-locate services to reduce bottlenecks introduced by network. The idea that a certain parallel process should be moved comes from an external service.
Speculation might not be effective here as I see, since the intent to move a container can be triggered hourly, daily, weekly or monthly - containers contain long running services, data pipelines. Or am I on the wrong line here? Can I simulate something like this with the current speculation?
Thanks for help!
On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com> wrote:
I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
Thanks,-Eric
From: Zoltán Zvara <zo...@gmail.com>
To: Vinod Kumar Vavilapalli <vi...@apache.org>
Cc: user@hadoop.apache.org
Sent: Wednesday, March 30, 2016 9:10 AM
Subject: Re: YARN re-locate container
How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
Re: YARN re-locate container
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
So, IIUC, there are two parts to what you need
(1) How do I schedule containers in the first place asking for locality?
(2) How do I keep my scheduling requirements the same even if some of my dependent service move / migrate.
For (1), YARN supports data-locality - you can give preferences for hosts / racks etc. More advanced ‘locality’ features are a work-in-progress.
As of now, we deem (2) to be too custom that we leave it up to the applications to orchestrate. Once we start putting more of the advanced locality features that I mentioned above, we can consider having the platform move containers automatically to keep those locality promises when services move/migrate.
HTH
+Vinod
> On Mar 31, 2016, at 1:44 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> The intent to move a container is to improve service-to-service locality, co-locate services to reduce bottlenecks introduced by network. The idea that a certain parallel process should be moved comes from an external service.
>
> Speculation might not be effective here as I see, since the intent to move a container can be triggered hourly, daily, weekly or monthly - containers contain long running services, data pipelines. Or am I on the wrong line here? Can I simulate something like this with the current speculation?
>
> Thanks for help!
>
> On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <eric.payne1000@yahoo.com <ma...@yahoo.com>> wrote:
> I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
>
> Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
>
> Thanks,
> -Eric
>
>
> From: Zoltán Zvara <zoltan.zvara@gmail.com <ma...@gmail.com>>
> To: Vinod Kumar Vavilapalli <vinodkv@apache.org <ma...@apache.org>>
> Cc: user@hadoop.apache.org <ma...@hadoop.apache.org>
> Sent: Wednesday, March 30, 2016 9:10 AM
> Subject: Re: YARN re-locate container
>
> How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.
> Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
>
> Am I understanding the architecture correctly here?
>
> On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vinodkv@apache.org <ma...@apache.org>> wrote:
> Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
>
> Are you looking for anything more specifically?
>
> +Vinod
>
> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zoltan.zvara@gmail.com <ma...@gmail.com>> wrote:
> >
> > Dear Hadoop Community,
> >
> > Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
> >
> > Thanks,
> > Zoltán
>
>
>
Re: YARN re-locate container
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
So, IIUC, there are two parts to what you need
(1) How do I schedule containers in the first place asking for locality?
(2) How do I keep my scheduling requirements the same even if some of my dependent service move / migrate.
For (1), YARN supports data-locality - you can give preferences for hosts / racks etc. More advanced ‘locality’ features are a work-in-progress.
As of now, we deem (2) to be too custom that we leave it up to the applications to orchestrate. Once we start putting more of the advanced locality features that I mentioned above, we can consider having the platform move containers automatically to keep those locality promises when services move/migrate.
HTH
+Vinod
> On Mar 31, 2016, at 1:44 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> The intent to move a container is to improve service-to-service locality, co-locate services to reduce bottlenecks introduced by network. The idea that a certain parallel process should be moved comes from an external service.
>
> Speculation might not be effective here as I see, since the intent to move a container can be triggered hourly, daily, weekly or monthly - containers contain long running services, data pipelines. Or am I on the wrong line here? Can I simulate something like this with the current speculation?
>
> Thanks for help!
>
> On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <eric.payne1000@yahoo.com <ma...@yahoo.com>> wrote:
> I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
>
> Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
>
> Thanks,
> -Eric
>
>
> From: Zoltán Zvara <zoltan.zvara@gmail.com <ma...@gmail.com>>
> To: Vinod Kumar Vavilapalli <vinodkv@apache.org <ma...@apache.org>>
> Cc: user@hadoop.apache.org <ma...@hadoop.apache.org>
> Sent: Wednesday, March 30, 2016 9:10 AM
> Subject: Re: YARN re-locate container
>
> How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.
> Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
>
> Am I understanding the architecture correctly here?
>
> On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vinodkv@apache.org <ma...@apache.org>> wrote:
> Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
>
> Are you looking for anything more specifically?
>
> +Vinod
>
> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zoltan.zvara@gmail.com <ma...@gmail.com>> wrote:
> >
> > Dear Hadoop Community,
> >
> > Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
> >
> > Thanks,
> > Zoltán
>
>
>
Re: YARN re-locate container
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
So, IIUC, there are two parts to what you need
(1) How do I schedule containers in the first place asking for locality?
(2) How do I keep my scheduling requirements the same even if some of my dependent service move / migrate.
For (1), YARN supports data-locality - you can give preferences for hosts / racks etc. More advanced ‘locality’ features are a work-in-progress.
As of now, we deem (2) to be too custom that we leave it up to the applications to orchestrate. Once we start putting more of the advanced locality features that I mentioned above, we can consider having the platform move containers automatically to keep those locality promises when services move/migrate.
HTH
+Vinod
> On Mar 31, 2016, at 1:44 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> The intent to move a container is to improve service-to-service locality, co-locate services to reduce bottlenecks introduced by network. The idea that a certain parallel process should be moved comes from an external service.
>
> Speculation might not be effective here as I see, since the intent to move a container can be triggered hourly, daily, weekly or monthly - containers contain long running services, data pipelines. Or am I on the wrong line here? Can I simulate something like this with the current speculation?
>
> Thanks for help!
>
> On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <eric.payne1000@yahoo.com <ma...@yahoo.com>> wrote:
> I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
>
> Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
>
> Thanks,
> -Eric
>
>
> From: Zoltán Zvara <zoltan.zvara@gmail.com <ma...@gmail.com>>
> To: Vinod Kumar Vavilapalli <vinodkv@apache.org <ma...@apache.org>>
> Cc: user@hadoop.apache.org <ma...@hadoop.apache.org>
> Sent: Wednesday, March 30, 2016 9:10 AM
> Subject: Re: YARN re-locate container
>
> How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.
> Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
>
> Am I understanding the architecture correctly here?
>
> On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vinodkv@apache.org <ma...@apache.org>> wrote:
> Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
>
> Are you looking for anything more specifically?
>
> +Vinod
>
> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zoltan.zvara@gmail.com <ma...@gmail.com>> wrote:
> >
> > Dear Hadoop Community,
> >
> > Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
> >
> > Thanks,
> > Zoltán
>
>
>
Re: YARN re-locate container
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
So, IIUC, there are two parts to what you need
(1) How do I schedule containers in the first place asking for locality?
(2) How do I keep my scheduling requirements the same even if some of my dependent service move / migrate.
For (1), YARN supports data-locality - you can give preferences for hosts / racks etc. More advanced ‘locality’ features are a work-in-progress.
As of now, we deem (2) to be too custom that we leave it up to the applications to orchestrate. Once we start putting more of the advanced locality features that I mentioned above, we can consider having the platform move containers automatically to keep those locality promises when services move/migrate.
HTH
+Vinod
> On Mar 31, 2016, at 1:44 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> The intent to move a container is to improve service-to-service locality, co-locate services to reduce bottlenecks introduced by network. The idea that a certain parallel process should be moved comes from an external service.
>
> Speculation might not be effective here as I see, since the intent to move a container can be triggered hourly, daily, weekly or monthly - containers contain long running services, data pipelines. Or am I on the wrong line here? Can I simulate something like this with the current speculation?
>
> Thanks for help!
>
> On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <eric.payne1000@yahoo.com <ma...@yahoo.com>> wrote:
> I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
>
> Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
>
> Thanks,
> -Eric
>
>
> From: Zoltán Zvara <zoltan.zvara@gmail.com <ma...@gmail.com>>
> To: Vinod Kumar Vavilapalli <vinodkv@apache.org <ma...@apache.org>>
> Cc: user@hadoop.apache.org <ma...@hadoop.apache.org>
> Sent: Wednesday, March 30, 2016 9:10 AM
> Subject: Re: YARN re-locate container
>
> How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.
> Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
>
> Am I understanding the architecture correctly here?
>
> On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vinodkv@apache.org <ma...@apache.org>> wrote:
> Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
>
> Are you looking for anything more specifically?
>
> +Vinod
>
> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zoltan.zvara@gmail.com <ma...@gmail.com>> wrote:
> >
> > Dear Hadoop Community,
> >
> > Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
> >
> > Thanks,
> > Zoltán
>
>
>
Re: YARN re-locate container
Posted by Zoltán Zvara <zo...@gmail.com>.
The intent to move a container is to improve service-to-service locality,
co-locate services to reduce bottlenecks introduced by network. The idea
that a certain parallel process should be moved comes from an external
service.
Speculation might not be effective here as I see, since the intent to move
a container can be triggered hourly, daily, weekly or monthly - containers
contain long running services, data pipelines. Or am I on the wrong line
here? Can I simulate something like this with the current speculation?
Thanks for help!
On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com>
wrote:
> I think it would help if I knew what the criteria is for wanting to move
> the container. In other words, was the container started on an undesirable
> node in the first place? Or, did the node become undesirable after the
> container started.
>
> Speculation could be considered a "move" operation for containers. If a
> container isn't finishing fast enough, the default speculator will start
> another container on a different node. Would it be possible to create a
> specialized speculator that understood your criteria for needing to move
> containers? If so, it could be done automatically / programatically.
>
> Thanks,
> -Eric
>
>
> ------------------------------
> *From:* Zoltán Zvara <zo...@gmail.com>
> *To:* Vinod Kumar Vavilapalli <vi...@apache.org>
> *Cc:* user@hadoop.apache.org
> *Sent:* Wednesday, March 30, 2016 9:10 AM
> *Subject:* Re: YARN re-locate container
>
> How is this achieved? As far as I see it now, after stopping a container,
> the AM must reallocate the same container with the same resource vector but
> with locality preferences pointed to the new, target node. After the new
> leash has been acquired, then the AM can take it to the new node and
> initiate a `startContainers` message.
> Our use-case with Ericsson would require a more simple API, where (for
> example) a `moveContainer` call from the AM would ask the RM or NM to move
> a container from one node to another (or to any of the specified set of
> preferred nodes). Move would simply kill the container and restart it on
> another node at any given time whenever it is possible - I feel questions
> around scheduling: how container moves should be handled? Probably not like
> simple allocations.
>
> Am I understanding the architecture correctly here?
>
> On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
> Containers can be restarted on other machines already today - YARN just
> leaves it up to the applications to do so.
>
> Are you looking for anything more specifically?
>
> +Vinod
>
> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com>
> wrote:
> >
> > Dear Hadoop Community,
> >
> > Is there any feature available, or on the road map to support the
> relocation of containers? (Simply restart the container on another machine.)
> >
> > Thanks,
> > Zoltán
>
>
>
>
Re: YARN re-locate container
Posted by Zoltán Zvara <zo...@gmail.com>.
The intent to move a container is to improve service-to-service locality,
co-locate services to reduce bottlenecks introduced by network. The idea
that a certain parallel process should be moved comes from an external
service.
Speculation might not be effective here as I see, since the intent to move
a container can be triggered hourly, daily, weekly or monthly - containers
contain long running services, data pipelines. Or am I on the wrong line
here? Can I simulate something like this with the current speculation?
Thanks for help!
On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com>
wrote:
> I think it would help if I knew what the criteria is for wanting to move
> the container. In other words, was the container started on an undesirable
> node in the first place? Or, did the node become undesirable after the
> container started.
>
> Speculation could be considered a "move" operation for containers. If a
> container isn't finishing fast enough, the default speculator will start
> another container on a different node. Would it be possible to create a
> specialized speculator that understood your criteria for needing to move
> containers? If so, it could be done automatically / programatically.
>
> Thanks,
> -Eric
>
>
> ------------------------------
> *From:* Zoltán Zvara <zo...@gmail.com>
> *To:* Vinod Kumar Vavilapalli <vi...@apache.org>
> *Cc:* user@hadoop.apache.org
> *Sent:* Wednesday, March 30, 2016 9:10 AM
> *Subject:* Re: YARN re-locate container
>
> How is this achieved? As far as I see it now, after stopping a container,
> the AM must reallocate the same container with the same resource vector but
> with locality preferences pointed to the new, target node. After the new
> leash has been acquired, then the AM can take it to the new node and
> initiate a `startContainers` message.
> Our use-case with Ericsson would require a more simple API, where (for
> example) a `moveContainer` call from the AM would ask the RM or NM to move
> a container from one node to another (or to any of the specified set of
> preferred nodes). Move would simply kill the container and restart it on
> another node at any given time whenever it is possible - I feel questions
> around scheduling: how container moves should be handled? Probably not like
> simple allocations.
>
> Am I understanding the architecture correctly here?
>
> On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
> Containers can be restarted on other machines already today - YARN just
> leaves it up to the applications to do so.
>
> Are you looking for anything more specifically?
>
> +Vinod
>
> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com>
> wrote:
> >
> > Dear Hadoop Community,
> >
> > Is there any feature available, or on the road map to support the
> relocation of containers? (Simply restart the container on another machine.)
> >
> > Thanks,
> > Zoltán
>
>
>
>
Re: YARN re-locate container
Posted by Zoltán Zvara <zo...@gmail.com>.
The intent to move a container is to improve service-to-service locality,
co-locate services to reduce bottlenecks introduced by network. The idea
that a certain parallel process should be moved comes from an external
service.
Speculation might not be effective here as I see, since the intent to move
a container can be triggered hourly, daily, weekly or monthly - containers
contain long running services, data pipelines. Or am I on the wrong line
here? Can I simulate something like this with the current speculation?
Thanks for help!
On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com>
wrote:
> I think it would help if I knew what the criteria is for wanting to move
> the container. In other words, was the container started on an undesirable
> node in the first place? Or, did the node become undesirable after the
> container started.
>
> Speculation could be considered a "move" operation for containers. If a
> container isn't finishing fast enough, the default speculator will start
> another container on a different node. Would it be possible to create a
> specialized speculator that understood your criteria for needing to move
> containers? If so, it could be done automatically / programatically.
>
> Thanks,
> -Eric
>
>
> ------------------------------
> *From:* Zoltán Zvara <zo...@gmail.com>
> *To:* Vinod Kumar Vavilapalli <vi...@apache.org>
> *Cc:* user@hadoop.apache.org
> *Sent:* Wednesday, March 30, 2016 9:10 AM
> *Subject:* Re: YARN re-locate container
>
> How is this achieved? As far as I see it now, after stopping a container,
> the AM must reallocate the same container with the same resource vector but
> with locality preferences pointed to the new, target node. After the new
> leash has been acquired, then the AM can take it to the new node and
> initiate a `startContainers` message.
> Our use-case with Ericsson would require a more simple API, where (for
> example) a `moveContainer` call from the AM would ask the RM or NM to move
> a container from one node to another (or to any of the specified set of
> preferred nodes). Move would simply kill the container and restart it on
> another node at any given time whenever it is possible - I feel questions
> around scheduling: how container moves should be handled? Probably not like
> simple allocations.
>
> Am I understanding the architecture correctly here?
>
> On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
> Containers can be restarted on other machines already today - YARN just
> leaves it up to the applications to do so.
>
> Are you looking for anything more specifically?
>
> +Vinod
>
> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com>
> wrote:
> >
> > Dear Hadoop Community,
> >
> > Is there any feature available, or on the road map to support the
> relocation of containers? (Simply restart the container on another machine.)
> >
> > Thanks,
> > Zoltán
>
>
>
>
Re: YARN re-locate container
Posted by Zoltán Zvara <zo...@gmail.com>.
The intent to move a container is to improve service-to-service locality,
co-locate services to reduce bottlenecks introduced by network. The idea
that a certain parallel process should be moved comes from an external
service.
Speculation might not be effective here as I see, since the intent to move
a container can be triggered hourly, daily, weekly or monthly - containers
contain long running services, data pipelines. Or am I on the wrong line
here? Can I simulate something like this with the current speculation?
Thanks for help!
On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <er...@yahoo.com>
wrote:
> I think it would help if I knew what the criteria is for wanting to move
> the container. In other words, was the container started on an undesirable
> node in the first place? Or, did the node become undesirable after the
> container started.
>
> Speculation could be considered a "move" operation for containers. If a
> container isn't finishing fast enough, the default speculator will start
> another container on a different node. Would it be possible to create a
> specialized speculator that understood your criteria for needing to move
> containers? If so, it could be done automatically / programatically.
>
> Thanks,
> -Eric
>
>
> ------------------------------
> *From:* Zoltán Zvara <zo...@gmail.com>
> *To:* Vinod Kumar Vavilapalli <vi...@apache.org>
> *Cc:* user@hadoop.apache.org
> *Sent:* Wednesday, March 30, 2016 9:10 AM
> *Subject:* Re: YARN re-locate container
>
> How is this achieved? As far as I see it now, after stopping a container,
> the AM must reallocate the same container with the same resource vector but
> with locality preferences pointed to the new, target node. After the new
> leash has been acquired, then the AM can take it to the new node and
> initiate a `startContainers` message.
> Our use-case with Ericsson would require a more simple API, where (for
> example) a `moveContainer` call from the AM would ask the RM or NM to move
> a container from one node to another (or to any of the specified set of
> preferred nodes). Move would simply kill the container and restart it on
> another node at any given time whenever it is possible - I feel questions
> around scheduling: how container moves should be handled? Probably not like
> simple allocations.
>
> Am I understanding the architecture correctly here?
>
> On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
> Containers can be restarted on other machines already today - YARN just
> leaves it up to the applications to do so.
>
> Are you looking for anything more specifically?
>
> +Vinod
>
> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com>
> wrote:
> >
> > Dear Hadoop Community,
> >
> > Is there any feature available, or on the road map to support the
> relocation of containers? (Simply restart the container on another machine.)
> >
> > Thanks,
> > Zoltán
>
>
>
>
Re: YARN re-locate container
Posted by Eric Payne <er...@yahoo.com.INVALID>.
I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
Thanks,-Eric
From: Zoltán Zvara <zo...@gmail.com>
To: Vinod Kumar Vavilapalli <vi...@apache.org>
Cc: user@hadoop.apache.org
Sent: Wednesday, March 30, 2016 9:10 AM
Subject: Re: YARN re-locate container
How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
Re: YARN re-locate container
Posted by Eric Payne <er...@yahoo.com.INVALID>.
I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
Thanks,-Eric
From: Zoltán Zvara <zo...@gmail.com>
To: Vinod Kumar Vavilapalli <vi...@apache.org>
Cc: user@hadoop.apache.org
Sent: Wednesday, March 30, 2016 9:10 AM
Subject: Re: YARN re-locate container
How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
Re: YARN re-locate container
Posted by Eric Payne <er...@yahoo.com.INVALID>.
I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
Thanks,-Eric
From: Zoltán Zvara <zo...@gmail.com>
To: Vinod Kumar Vavilapalli <vi...@apache.org>
Cc: user@hadoop.apache.org
Sent: Wednesday, March 30, 2016 9:10 AM
Subject: Re: YARN re-locate container
How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
Re: YARN re-locate container
Posted by Eric Payne <er...@yahoo.com.INVALID>.
I think it would help if I knew what the criteria is for wanting to move the container. In other words, was the container started on an undesirable node in the first place? Or, did the node become undesirable after the container started.
Speculation could be considered a "move" operation for containers. If a container isn't finishing fast enough, the default speculator will start another container on a different node. Would it be possible to create a specialized speculator that understood your criteria for needing to move containers? If so, it could be done automatically / programatically.
Thanks,-Eric
From: Zoltán Zvara <zo...@gmail.com>
To: Vinod Kumar Vavilapalli <vi...@apache.org>
Cc: user@hadoop.apache.org
Sent: Wednesday, March 30, 2016 9:10 AM
Subject: Re: YARN re-locate container
How is this achieved? As far as I see it now, after stopping a container, the AM must reallocate the same container with the same resource vector but with locality preferences pointed to the new, target node. After the new leash has been acquired, then the AM can take it to the new node and initiate a `startContainers` message.Our use-case with Ericsson would require a more simple API, where (for example) a `moveContainer` call from the AM would ask the RM or NM to move a container from one node to another (or to any of the specified set of preferred nodes). Move would simply kill the container and restart it on another node at any given time whenever it is possible - I feel questions around scheduling: how container moves should be handled? Probably not like simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
Re: YARN re-locate container
Posted by Zoltán Zvara <zo...@gmail.com>.
How is this achieved? As far as I see it now, after stopping a container,
the AM must reallocate the same container with the same resource vector but
with locality preferences pointed to the new, target node. After the new
leash has been acquired, then the AM can take it to the new node and
initiate a `startContainers` message.
Our use-case with Ericsson would require a more simple API, where (for
example) a `moveContainer` call from the AM would ask the RM or NM to move
a container from one node to another (or to any of the specified set of
preferred nodes). Move would simply kill the container and restart it on
another node at any given time whenever it is possible - I feel questions
around scheduling: how container moves should be handled? Probably not like
simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org>
wrote:
> Containers can be restarted on other machines already today - YARN just
> leaves it up to the applications to do so.
>
> Are you looking for anything more specifically?
>
> +Vinod
>
> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com>
> wrote:
> >
> > Dear Hadoop Community,
> >
> > Is there any feature available, or on the road map to support the
> relocation of containers? (Simply restart the container on another machine.)
> >
> > Thanks,
> > Zoltán
>
>
Re: YARN re-locate container
Posted by Zoltán Zvara <zo...@gmail.com>.
How is this achieved? As far as I see it now, after stopping a container,
the AM must reallocate the same container with the same resource vector but
with locality preferences pointed to the new, target node. After the new
leash has been acquired, then the AM can take it to the new node and
initiate a `startContainers` message.
Our use-case with Ericsson would require a more simple API, where (for
example) a `moveContainer` call from the AM would ask the RM or NM to move
a container from one node to another (or to any of the specified set of
preferred nodes). Move would simply kill the container and restart it on
another node at any given time whenever it is possible - I feel questions
around scheduling: how container moves should be handled? Probably not like
simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org>
wrote:
> Containers can be restarted on other machines already today - YARN just
> leaves it up to the applications to do so.
>
> Are you looking for anything more specifically?
>
> +Vinod
>
> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com>
> wrote:
> >
> > Dear Hadoop Community,
> >
> > Is there any feature available, or on the road map to support the
> relocation of containers? (Simply restart the container on another machine.)
> >
> > Thanks,
> > Zoltán
>
>
Re: YARN re-locate container
Posted by Zoltán Zvara <zo...@gmail.com>.
How is this achieved? As far as I see it now, after stopping a container,
the AM must reallocate the same container with the same resource vector but
with locality preferences pointed to the new, target node. After the new
leash has been acquired, then the AM can take it to the new node and
initiate a `startContainers` message.
Our use-case with Ericsson would require a more simple API, where (for
example) a `moveContainer` call from the AM would ask the RM or NM to move
a container from one node to another (or to any of the specified set of
preferred nodes). Move would simply kill the container and restart it on
another node at any given time whenever it is possible - I feel questions
around scheduling: how container moves should be handled? Probably not like
simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org>
wrote:
> Containers can be restarted on other machines already today - YARN just
> leaves it up to the applications to do so.
>
> Are you looking for anything more specifically?
>
> +Vinod
>
> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com>
> wrote:
> >
> > Dear Hadoop Community,
> >
> > Is there any feature available, or on the road map to support the
> relocation of containers? (Simply restart the container on another machine.)
> >
> > Thanks,
> > Zoltán
>
>
Re: YARN re-locate container
Posted by Zoltán Zvara <zo...@gmail.com>.
How is this achieved? As far as I see it now, after stopping a container,
the AM must reallocate the same container with the same resource vector but
with locality preferences pointed to the new, target node. After the new
leash has been acquired, then the AM can take it to the new node and
initiate a `startContainers` message.
Our use-case with Ericsson would require a more simple API, where (for
example) a `moveContainer` call from the AM would ask the RM or NM to move
a container from one node to another (or to any of the specified set of
preferred nodes). Move would simply kill the container and restart it on
another node at any given time whenever it is possible - I feel questions
around scheduling: how container moves should be handled? Probably not like
simple allocations.
Am I understanding the architecture correctly here?
On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli <vi...@apache.org>
wrote:
> Containers can be restarted on other machines already today - YARN just
> leaves it up to the applications to do so.
>
> Are you looking for anything more specifically?
>
> +Vinod
>
> > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com>
> wrote:
> >
> > Dear Hadoop Community,
> >
> > Is there any feature available, or on the road map to support the
> relocation of containers? (Simply restart the container on another machine.)
> >
> > Thanks,
> > Zoltán
>
>
Re: YARN re-locate container
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: YARN re-locate container
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: YARN re-locate container
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: YARN re-locate container
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Containers can be restarted on other machines already today - YARN just leaves it up to the applications to do so.
Are you looking for anything more specifically?
+Vinod
> On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <zo...@gmail.com> wrote:
>
> Dear Hadoop Community,
>
> Is there any feature available, or on the road map to support the relocation of containers? (Simply restart the container on another machine.)
>
> Thanks,
> Zoltán
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org