Anti-affinity

[Krishna Kishore Bonagiri <wr...@gmail.com>]

Hi,

  Is there any plan to implement anti-affinity? I know it would obviously
depend on the support for it from YARN. Can it be expected in the near
feature? Do you guys know if YARN has any plans for it?

Thanks,
Ksihore

Re: Anti-affinity

[Jakob Homan <jg...@gmail.com>]

We ran into lots of problems with this in Samza:
https://issues.apache.org/jira/browse/SAMZA-124 .  Hopefully you'll
have better luck.

On 17 March 2015 at 14:41, Thomas Weise <th...@gmail.com> wrote:
> Absent YARN support, would it make sense to add logic in the Slider AM to
> ensures anti-affinity. It would be possible to do this by requesting
> specific hosts, if the initial AM response does not return all containers
> on different hosts.
>
> Thanks,
> Thomas
>
>
> On Tue, Mar 17, 2015 at 11:05 AM, Ted Yu <yu...@gmail.com> wrote:
>
>> Support from YARN is needed.
>>
>> Please see https://issues.apache.org/jira/browse/YARN-1042
>>
>> Cheers
>>
>> On Tue, Mar 17, 2015 at 1:56 AM, Krishna Kishore Bonagiri <
>> write2kishore@gmail.com> wrote:
>>
>> > Hi,
>> >
>> >   Is there any plan to implement anti-affinity? I know it would obviously
>> > depend on the support for it from YARN. Can it be expected in the near
>> > feature? Do you guys know if YARN has any plans for it?
>> >
>> > Thanks,
>> > Ksihore
>> >
>>

Re: Anti-affinity

[Steve Loughran <st...@hortonworks.com>]

> On 17 Mar 2015, at 21:41, Thomas Weise <th...@gmail.com> wrote:
> 
> Absent YARN support, would it make sense to add logic in the Slider AM to
> ensures anti-affinity.

+1


> It would be possible to do this by requesting
> specific hosts, if the initial AM response does not return all containers
> on different hosts.
> 

I'm doing the SLIDER-799 escalation code right now, primarily so that the AM doesn't ask for relaxed placement until a time period has expired. It'll use the persisted placement history to select those first locations (skipping ones considered recently unreliable)

I suppose we could extend that to an initial startup (or any situation where you want more instances than you have any record of) to pick some at random & escalate if unmet.

...feature creep for a Placement Phase III JIRA, I suspect...

Re: Anti-affinity

[Thomas Weise <th...@gmail.com>]

Absent YARN support, would it make sense to add logic in the Slider AM to
ensures anti-affinity. It would be possible to do this by requesting
specific hosts, if the initial AM response does not return all containers
on different hosts.

Thanks,
Thomas


On Tue, Mar 17, 2015 at 11:05 AM, Ted Yu <yu...@gmail.com> wrote:

> Support from YARN is needed.
>
> Please see https://issues.apache.org/jira/browse/YARN-1042
>
> Cheers
>
> On Tue, Mar 17, 2015 at 1:56 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
> > Hi,
> >
> >   Is there any plan to implement anti-affinity? I know it would obviously
> > depend on the support for it from YARN. Can it be expected in the near
> > feature? Do you guys know if YARN has any plans for it?
> >
> > Thanks,
> > Ksihore
> >
>

Re: Anti-affinity

[Ted Yu <yu...@gmail.com>]

Support from YARN is needed.

Please see https://issues.apache.org/jira/browse/YARN-1042

Cheers

On Tue, Mar 17, 2015 at 1:56 AM, Krishna Kishore Bonagiri <
write2kishore@gmail.com> wrote:

> Hi,
>
>   Is there any plan to implement anti-affinity? I know it would obviously
> depend on the support for it from YARN. Can it be expected in the near
> feature? Do you guys know if YARN has any plans for it?
>
> Thanks,
> Ksihore
>