Affinity support in Apex

[Devendra Tagare <de...@datatorrent.com>]

Hi All,

I was checking out affinity support in a DAG that has 3 operators,

Operator 1 -> Operator 2 -> Operator 3

I am trying to ensure that Operator 1 and Operator 3 are on the same node
always and Operator 2 could be anywhere.

Here's the relevant code snippet,

    List<String> operatorList = new LinkedList<String>();

    operatorList.add("Operator1");

    operatorList.add("Operator3");

    AffinityRulesSet ruleSet = new AffinityRulesSet();

    List<AffinityRule> rules = new ArrayList<>();

    rules.add(new AffinityRule(Type.AFFINITY, operatorList, Locality.
NODE_LOCAL, false));

    ruleSet.setAffinityRules(rules);

    dag.setAttribute(DAGContext.AFFINITY_RULES_SET, ruleSet);


This seems to be working consistently for 2 operators but does not work for
3 operators with Locality.NODE_LOCAL.

I have set relaxLocality to false so it should be enforced always.

It works for 3 operators when Locality is CONTAINER_LOCAL or THREAD_LOCAL.

Is this the expected behavior  ?

Thanks,

Dev