You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@slider.apache.org by "David.Serafini" <Da...@target.com> on 2017/12/04 23:29:48 UTC
Re: [EXTERNAL] Re: Ensure no more than one container for the same
application is running in the same host
That's great for people on the bleeding edge, but I don't expect our production Hadoop cluster will migrate to 3.x any time soon, so I'll continue to run slider 0.9x
-david
On 12/4/17, 3:26 PM, Gour Saha <gs...@hortonworks.com> wrote:
You are right. Since YARN did not implement anti-affinity, Slider
implemented it in its own way. No special YARN feature was used to
implement Slider¹s anti-affinity. Which hadoop version are you using?
By the way, I hope you are following
https://issues.apache.org/jira/browse/YARN-4692 - Slider is being
deprecated and has been rolled into YARN as a first class module (already
in trunk now and most likely the first release will be 3.1.x). It is being
referred to as ³YARN Service² now -
https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn
/hadoop-yarn-site/src/site/markdown/yarn-service/QuickStart.md
-Gour
On 12/4/17, 3:09 PM, "Juan Rodríguez Hortalá"
<ju...@gmail.com> wrote:
>One more question, in
>https://slider.incubator.apache.org/docs/configuration/resources.html#plac
>ement
>it says that "Once YARN adds support for this, Slider will support it."
>and
>https://issues.apache.org/jira/browse/YARN-1042 for "add ability to
>specify
>affinity/anti-affinity in container requests" is still unresolved. However
>in I see ANTI_AFFINITY_REQUIRED in
>https://github.com/apache/incubator-slider/blob/develop/slider-core/src/ma
>in/java/org/apache/slider/providers/PlacementPolicy.java#L57
>and other places of the code. Is anti affinity implemented in slider
>before
>YARN-1042, by other means independent from that YARN feature? Is there a
>minimal version of YARN required to use anti affinity in Slider?
>
>Thanks again,
>
>Juan
>
>On Mon, Dec 4, 2017 at 2:56 PM, Juan Rodríguez Hortalá <
>juan.rodriguez.hortala@gmail.com> wrote:
>
>> Thanks a lot Gour! I'll do so.
>>
>> Greetings,
>>
>> Juan
>>
>> On Fri, Dec 1, 2017 at 11:38 PM, Gour Saha <gs...@hortonworks.com>
>>wrote:
>>
>>> Set yarn.component.placement.policy to 4 for the component in
>>> resources.json to get anti- affinity.
>>>
>>> -Gour
>>>
>>> - Sent from my iPhone
>>>
>>> On Dec 1, 2017, at 7:14 PM, Juan Rodr?guez Hortal? <
>>>
>>>juan.rodriguez.hortala@gmail.com<mailto:juan.rodriguez.hortala@gmail.com
>>>>>
>>> wrote:
>>>
>>> Hi,
>>>
>>> I'm new to slider, and I'm trying to run an application for a
>>>distributed
>>> database on slider using simple packaging. I have to specify a RPC port
>>> that should be the same in all daemons running in each node. The
>>>problem
>>> is
>>> that if 2 containers for this application are running on the same node
>>> then
>>> they would try to use the same port. It is possible to specify somehow
>>> that
>>> we don't want to have more than one container for a specific component
>>>of
>>> the application running in the same node?
>>>
>>> Thanks in advance.
>>>
>>> Juan
>>>
>>
>>
Re: [EXTERNAL] Re: Ensure no more than one container for the same
application is running in the same host
Posted by Juan Rodríguez Hortalá <ju...@gmail.com>.
For now we are running Hadoop 2.7, but this reference will be very useful
for the future.
Thanks a lot!
On Mon, Dec 4, 2017 at 3:29 PM, David.Serafini <Da...@target.com>
wrote:
> That's great for people on the bleeding edge, but I don't expect our
> production Hadoop cluster will migrate to 3.x any time soon, so I'll
> continue to run slider 0.9x
>
> -david
>
>
> On 12/4/17, 3:26 PM, Gour Saha <gs...@hortonworks.com> wrote:
>
> You are right. Since YARN did not implement anti-affinity, Slider
> implemented it in its own way. No special YARN feature was used to
> implement Slider¹s anti-affinity. Which hadoop version are you using?
>
> By the way, I hope you are following
> https://issues.apache.org/jira/browse/YARN-4692 - Slider is being
> deprecated and has been rolled into YARN as a first class module
> (already
> in trunk now and most likely the first release will be 3.1.x). It is
> being
> referred to as ³YARN Service² now -
> https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-
> project/hadoop-yarn
> /hadoop-yarn-site/src/site/markdown/yarn-service/QuickStart.md
>
> -Gour
>
> On 12/4/17, 3:09 PM, "Juan Rodríguez Hortalá"
> <ju...@gmail.com> wrote:
>
> >One more question, in
> >https://slider.incubator.apache.org/docs/configuration/
> resources.html#plac
> >ement
> >it says that "Once YARN adds support for this, Slider will support
> it."
> >and
> >https://issues.apache.org/jira/browse/YARN-1042 for "add ability to
> >specify
> >affinity/anti-affinity in container requests" is still unresolved.
> However
> >in I see ANTI_AFFINITY_REQUIRED in
> >https://github.com/apache/incubator-slider/blob/develop/
> slider-core/src/ma
> >in/java/org/apache/slider/providers/PlacementPolicy.java#L57
> >and other places of the code. Is anti affinity implemented in slider
> >before
> >YARN-1042, by other means independent from that YARN feature? Is
> there a
> >minimal version of YARN required to use anti affinity in Slider?
> >
> >Thanks again,
> >
> >Juan
> >
> >On Mon, Dec 4, 2017 at 2:56 PM, Juan Rodríguez Hortalá <
> >juan.rodriguez.hortala@gmail.com> wrote:
> >
> >> Thanks a lot Gour! I'll do so.
> >>
> >> Greetings,
> >>
> >> Juan
> >>
> >> On Fri, Dec 1, 2017 at 11:38 PM, Gour Saha <gs...@hortonworks.com>
> >>wrote:
> >>
> >>> Set yarn.component.placement.policy to 4 for the component in
> >>> resources.json to get anti- affinity.
> >>>
> >>> -Gour
> >>>
> >>> - Sent from my iPhone
> >>>
> >>> On Dec 1, 2017, at 7:14 PM, Juan Rodr?guez Hortal? <
> >>>
> >>>juan.rodriguez.hortala@gmail.com<mailto:juan.
> rodriguez.hortala@gmail.com
> >>>>>
> >>> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I'm new to slider, and I'm trying to run an application for a
> >>>distributed
> >>> database on slider using simple packaging. I have to specify a RPC
> port
> >>> that should be the same in all daemons running in each node. The
> >>>problem
> >>> is
> >>> that if 2 containers for this application are running on the same
> node
> >>> then
> >>> they would try to use the same port. It is possible to specify
> somehow
> >>> that
> >>> we don't want to have more than one container for a specific
> component
> >>>of
> >>> the application running in the same node?
> >>>
> >>> Thanks in advance.
> >>>
> >>> Juan
> >>>
> >>
> >>
>
>
>
>
>