[jira] [Created] (MESOS-2602) Provide a way to "push" cluster state updates to a registered service.

["Christos Kozyrakis (JIRA)" <ji...@apache.org>]

Christos Kozyrakis created MESOS-2602:
-----------------------------------------

             Summary: Provide a way to "push" cluster state updates to a registered service. 
                 Key: MESOS-2602
                 URL: https://issues.apache.org/jira/browse/MESOS-2602
             Project: Mesos
          Issue Type: Improvement
            Reporter: Christos Kozyrakis


At the moment, service discovery systems like Mesos-DNS must periodically pull the cluster state through state.json. This is extremely wasteful and will not scale to large clusters. If the polling interval is low, the master load will grow significantly. If the polling interval is high, there will be added latency to service discovery. Moreover, the way state.json works right now, one keeps reading the same information over and over again, including info about about tasks no longer running. 

We can design an endpoint that allows a "push" approach for state information. Here is one of the possible ways to set it up:
- a service can hit the end point at (re)start to get information for all currently running tasks. 
- a service can also register itself to get receive updates to task state beyond that (ie, notifications of tasks starting/ending/etc). We may want to add some qualifiers here, since service discovery systems care only about certain types of updates.  
This can be implemented through direct messaging, through a message queue, by putting messages in etcd/zookeeper, etc. We should pick the way that is most scalable. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)