[jira] Commented: (HADOOP-4383) Add standard interface/methods for all services to query IPC and HTTP addresses and ports

["Todd Lipcon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713322#action_12713322 ] 

Todd Lipcon commented on HADOOP-4383:
-------------------------------------

Rather than adding interfaces to the specific daemons, I'd like to propose going the other direction and factoring this into a ServiceLocator interface. This would provide the traditional "name service" role which many distributed systems implicitly assume exists.

When any daemon opens up a server, they would register it (IP(s), port, protocol type) with the ServiceLocator. When any daemon (or client) wants to connect to a specific endpoint, they query the ServiceLocator to find it. The initial implementation of this interface would simply perform lookups in Configuration, maintaining the status quo, but I foresee a lot other very useful potential implementations:
 - The J2EE-ish solution (I'm not a big J2EE guy, but I think JMS or JNDI are the appropriate TLAs here?)
 - ZooKeeper
 - mdns (aka zeroconf)
 - RFC 2136 dynamic DNS updates
 - Organization specific service locators (eg SmartFrog)
 - Amazon Elastic IP (eg automatically attach an elastic IP to the NN when the NN boots)

Making this nicely pluggable through contrib jars would do well to allow flexibility while keeping core clean.

This should solve several goals in parallel:
  - Factors out common code regarding "bind address" configurations, wildcard addresses, localhost vs wildcard vs external IPs, etc
  - Reduces the reliance on "writing back" into Conf objects at service start time, which I think most people would agree is a somewhat dirty practice.
  - Provides pluggable methods we'll need later if we look into automatic failover of master daemons
  - Provides better integration with external systems already in use in various organizations (eg SmartFrog, Thrift-based service directories, etc)

> Add standard interface/methods for all services to query IPC and HTTP addresses and ports
> -----------------------------------------------------------------------------------------
>
>                 Key: HADOOP-4383
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4383
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs, mapred
>    Affects Versions: 0.20.0
>            Reporter: Steve Loughran
>            Priority: Minor
>
> This is something I've ended up doing in subclasses of all the services: methods to get at the IPC and HTTP port and addresses. Some services have exported methods for this (JobTracker), others package-private member variables (namenode), while others don't allow you to get at all the data (Datanode keeps the http server private). 
> A uniform way to query any service for its live port and address values make some aspects of service management much easier, such as feeding those values in to http page monitoring tools.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.