You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by Jung JaeHwa <bl...@apache.org> on 2014/06/26 04:10:45 UTC

Review Request 22997: TAJO-611: (Umbrella) Service Discovery

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22997/
-----------------------------------------------------------

Review request for Tajo.


Bugs: TAJO-611
    https://issues.apache.org/jira/browse/TAJO-611


Repository: tajo


Description
-------

As we talked offline, high availability is one of our next goal. Service discovery can help us maintain health statuses for all daemons ( master and workers) . Meanwhile, those daemons can find each other easily. Furthermore, it's very useful for my current work TAJO-540 cuz it can randomly select nodes for tajo scheduler.
One of the best candidates is Netflix Curator. http://curator.apache.org/curator-x-discovery/index.html
I'd like to introduce Xuhui to help us with this issue. Xuhui was my colleague at Alibaba group. He was active in Hive community, one of his job is adding multi-distinct aggregation feature to Hive. Currently, he is a researcher work at Microsoft.
Mafish
Below is comment on this issue from Xuhui
To my understanding, this feature is for high availability as well as high scalability. We don't need to provide all machine info for every service when tajo starts. Instead, we can dynamically register services to service discovery if necessary. Also, if some machine fails, it can be easily detected and replaced.


Diffs
-----

  tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java 96a2691 
  tajo-core/pom.xml 32ae266 
  tajo-core/src/main/java/org/apache/tajo/curator/CuratorConstants.java PRE-CREATION 
  tajo-core/src/main/java/org/apache/tajo/curator/ServiceInstanceSerializer.java PRE-CREATION 
  tajo-core/src/main/java/org/apache/tajo/curator/zk/ZKCurator.java PRE-CREATION 
  tajo-core/src/main/proto/CuratorProtocol.proto PRE-CREATION 
  tajo-core/src/test/java/org/apache/tajo/curator/zk/TestZKCurator.java PRE-CREATION 

Diff: https://reviews.apache.org/r/22997/diff/


Testing
-------

mvn clean install


Thanks,

Jung JaeHwa