[GitHub] samza pull request #471: SAMZA-1649: Improve host-aware allocation to accoun...

[vjagadish <gi...@git.apache.org>]

GitHub user vjagadish opened a pull request:

    https://github.com/apache/samza/pull/471

    SAMZA-1649: Improve host-aware allocation to account for strict locality

    Working on a doc for the behavior of the CapacityScheduler and further testing is needed on an actual cluster - but here's a summary: 
     - Node-local requests are honored only when relax-locality = false
     - With relax-locality = true, the scheduler biases interactivity over data-locality for requests that ask for few resources relative to the size of the cluster. 
    
    In addition to the above change, this PR also modifies the allocator algorithm to fallback to "ANY_HOST" requests so that we make progress when the node is unavailable.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/vjagadish1989/samza relax-locality-fix

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/samza/pull/471.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #471
    
----
commit a117356b59065a50f9bbffacacfb799d3ff2dc9f
Author: Jagadish <jv...@...>
Date:   2018-04-11T21:33:52Z

    Initial patch for relax-locality

commit 002cb87d02fef3fee73439daee36074402675484
Author: Jagadish <jv...@...>
Date:   2018-04-12T00:22:05Z

    initial stab at an unit test

commit 7ce9e9457e97280a8b09ad6684d22079fe527f12
Author: Jagadish <jv...@...>
Date:   2018-04-12T07:10:20Z

    Add more assertions for ensuring containers get released

commit 2e51d378053b1f4df1e0892cbcdeaeae02f78d81
Author: Jagadish <jv...@...>
Date:   2018-04-12T15:37:21Z

    Improve documentation and refactor the testExpiredContainers

commit 5ceafc5a50ab3874286889c68df4660b06237472
Author: Jagadish <jv...@...>
Date:   2018-04-12T16:15:04Z

    Minor clean-ups: remove deprecated usages

commit f5b9d566879f34f813e65da0a2e0152d4470b8d4
Author: Jagadish <jv...@...>
Date:   2018-04-12T16:27:31Z

    Remove additional debug logging. Simulate a cluster manager timeout

----


---

[GitHub] samza pull request #471: SAMZA-1649: Improve host-aware allocation to accoun...

[asfgit <gi...@git.apache.org>]

Github user asfgit closed the pull request at:

    https://github.com/apache/samza/pull/471


---