You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@slider.apache.org by Steve Loughran <st...@hortonworks.com> on 2014/08/08 20:29:10 UTC

slider-302 -> moving to a queue based model for triggering AM actions

when you next check out develop/ , you'll see a new package in the am,
org.apache.slider.server.appmaster.actions.

I've added a queue system which bits of the system should use to queue
instructions to the AM, rather than do it synchronized. This is where I
plan to consolidate cluster change events (flexing, container-loss
notification), as well as AM lifecycle events (stop). This reduces the
amount of sync complexity -threads like the Agent heartbeat thread just
push out a ProviderReportedContainerLoss event and don't wait for a reply.


   1. We shouldn't be calling the AM directly from things like: app state,
   provider, container launcher -instead queue AsyncActions which do the
   invocation.
   2. As well as the basic queue, there's a delayed action queue alongside:
   things put in there are delayed until their time expires, at which point
   they go into the tail of the main queue. This lets us schedule things for
   "later". As an example, I'm going to implement sliding window resets by
   having an event scheduled 60 minutes later, and then have it reschedule
   itself 60 minutes after it finishes.
   3. I'm thinking about whether in future we could scan the queue and
   postpone certain events (cluster review and refresh) if the queue after it
   has any operation that is marked as changing cluster state (Adding/removing
   nodes, halting app itself). This will avoid needless cluster manipulation
   operations.

Ultimately this (somewhat actor based) model is cleaner, I'm just moving to
it carefully now that the core is there.

As usual if things break complain with a JIRA related to SLIDER-1

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.