Hadoop 1.0 Interface Classification Discussion

[Sanjay Radia <sr...@yahoo-inc.com>]

One of the tasks/prerequisites  for Hadoop 1.0 is to decide which  
interfaces are
  external facing and which are internal facing.

   An interface declared java-public is not necessarily a hadoop
public/external interface.  Instead I use the terms external and  
internal to
avoid confusion with a java class's public and private tag.
What does external and internal mean: Any application can use Hadoop's  
external interface;
however, they should not use any hadoop internal interfaces.

   Further it is helpful to classify the stability levels of interfaces.
   I have used a interface taxonomy that I have derived from OpenSolaris
      (http://redirect.corp.yahoo.com/?url=http%3A%2F%2Fwww.opensolaris.org%2Fos%2Fcommunity%2Farc%2Fpolicies%2Finterface-taxonomy%2F%23Advice 
)

-------------
Release terminology used below:
Standard release numbering:
- Only bug fixes in dot releases: m.x.y
	- no changes to API, disk format, protocols or config etc.
- new features in major (m.0) and minor (m.x.0) releases
-------------

    Here is a *strawman* breakdown of our various interfaces along  
with their
    visibility and stability.
    There is sure to be debate over this. Is this classification useful?
    What interfaces are not covered below?

    * External-stable (can break compatibility at major releases)
       - Filesystem, MapReduce, Config, shell CLI and shell CLI output
       - Mapred.lib (stable after cleanup)
       - HDFS-JMX MBeans?

    * External-evolving (can break compatibility at minor releases)
		(Evolving does not mean that we will break compatibility for trivial  
reasons
          at a minor release; we have the freedom to break compatibility
          when there is good reason.)
       - Core APIs that we are not ready to declare as stable: RPC and  
IPC APIs , Metrics,
              - alternatively classify  these core APIs as Internal- 
Evolving
       - Job logs - should this be stable ?

    * External-Not an interface
       - Hadoop Web GUI

    * Internal-stable - (can break compatibility at major releases)
       - Intra HDFS and MR protocols (rolling upgrades in the future)

    * Internal-evolving (can break compatibility at minor releases)
       - TT-to-task umbilical cord
    * Internal-private
        - System logs, etc.
        - The rest of the Hadoop classes and interfaces.

We need to remove all internal interfaces from the hadoop-external  
javadoc.
     Add javadoc tag to interfaces to reflect stability?