[GitHub] markusthoemmes opened a new pull request #3240: Add a loadbalancer with local state and horizontal invoker sharding.

[GitBox <gi...@apache.org>]

markusthoemmes opened a new pull request #3240: Add a loadbalancer with local state and horizontal invoker sharding.
URL: https://github.com/apache/incubator-openwhisk/pull/3240
 
 
   The current ContainerPoolBalancer suffers a couple of problems and bottlenecks:
   
   1. **Inconsistent state:** The data-structures keeping the state for that loadbalancer are not thread-safely handled, meaning there can be queuing to some invokers even though there is free capacity on other invokers.
   2. **Asynchronously shared state:** Sharing the state is needed for a high-available deployment of multiple controllers and for horizontal scale in those. Said state-sharing makes point 1 even worse and isn't anywhere fast enough to be able to efficiently schedule quick bursts.
   3. **Bottlenecks:** Getting the state from the outside (like for the ActivationThrottle) is a very costly operation (at least in the sharedstate case) and actually bottlenecks the whole invocation path. Getting the current state of the invokers is a second bottleneck, where one request is made to the corresponding actor for each invocation.
   
   This new implementation aims to solve the problems mentioned above as follows:
   
   1. **All state is local:** There is no shared state. Resources are managed through horizontal sharding. Horizontal sharding means: The invokers' slots are evenly divided between the loadbalancers in existence. If we deploy 2 loadbalancers and each invoker has 16 slots, each of the loadbalancers will have access to 8 slots on each invoker.
   2. **Slots are given away atomically:** When scheduling an activation, the slot is immediatly assigned to that activation (implemented through Semaphores). That means: Even in concurrent schedules, there will not be an overload on an invoker as long as there is capacity left on that invoker.
   3. **Asynchronous updates of slow data:** Slowly changing data, like a change in the invoker's state, is asynchronously handled and updated to a local version of the state. Querying the state is as cheap as it can be.
   
   ### TODO
   
   - [ ] Tests (mainly for the state updates and the scheduler)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services