You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by ac...@apache.org on 2013/06/26 08:10:06 UTC
svn commit: r1496760 - in
/hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common:
./ src/main/proto/ src/site/apt/
Author: acmurthy
Date: Wed Jun 26 06:10:05 2013
New Revision: 1496760
URL: http://svn.apache.org/r1496760
Log:
Merged HADOOP-9619 svn merge -c 1495564
Modified:
hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/CHANGES.txt
hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/HAServiceProtocol.proto
hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/IpcConnectionContext.proto
hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/ProtobufRpcEngine.proto
hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/ProtocolInfo.proto
hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto
hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/Security.proto
hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/ZKFCProtocol.proto
hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/site/apt/Compatibility.apt.vm
Modified: hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/CHANGES.txt?rev=1496760&r1=1496759&r2=1496760&view=diff
==============================================================================
--- hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/CHANGES.txt (original)
+++ hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/CHANGES.txt Wed Jun 26 06:10:05 2013
@@ -116,6 +116,8 @@ Release 2.1.0-beta - UNRELEASED
HADOOP-8608. Add Configuration API for parsing time durations. (cdouglas)
+ HADOOP-9619 Mark stability of .proto files (sanjay Radia)
+
OPTIMIZATIONS
HADOOP-9150. Avoid unnecessary DNS resolution attempts for logical URIs
Modified: hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/HAServiceProtocol.proto
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/HAServiceProtocol.proto?rev=1496760&r1=1496759&r2=1496760&view=diff
==============================================================================
--- hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/HAServiceProtocol.proto (original)
+++ hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/HAServiceProtocol.proto Wed Jun 26 06:10:05 2013
@@ -16,6 +16,12 @@
* limitations under the License.
*/
+/**
+ * These .proto interfaces are private and stable.
+ * Please see http://wiki.apache.org/hadoop/Compatibility
+ * for what changes are allowed for a *stable* .proto interface.
+ */
+
option java_package = "org.apache.hadoop.ha.proto";
option java_outer_classname = "HAServiceProtocolProtos";
option java_generic_services = true;
Modified: hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/IpcConnectionContext.proto
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/IpcConnectionContext.proto?rev=1496760&r1=1496759&r2=1496760&view=diff
==============================================================================
--- hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/IpcConnectionContext.proto (original)
+++ hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/IpcConnectionContext.proto Wed Jun 26 06:10:05 2013
@@ -15,6 +15,13 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+
+/**
+ * These .proto interfaces are private and stable.
+ * Please see http://wiki.apache.org/hadoop/Compatibility
+ * for what changes are allowed for a *stable* .proto interface.
+ */
+
option java_package = "org.apache.hadoop.ipc.protobuf";
option java_outer_classname = "IpcConnectionContextProtos";
option java_generate_equals_and_hash = true;
Modified: hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/ProtobufRpcEngine.proto
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/ProtobufRpcEngine.proto?rev=1496760&r1=1496759&r2=1496760&view=diff
==============================================================================
--- hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/ProtobufRpcEngine.proto (original)
+++ hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/ProtobufRpcEngine.proto Wed Jun 26 06:10:05 2013
@@ -17,6 +17,12 @@
*/
/**
+ * These .proto interfaces are private and stable.
+ * Please see http://wiki.apache.org/hadoop/Compatibility
+ * for what changes are allowed for a *stable* .proto interface.
+ */
+
+/**
* These are the messages used by Hadoop RPC for the Rpc Engine Protocol Buffer
* to marshal the request and response in the RPC layer.
* The messages are sent in addition to the normal RPC header as
Modified: hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/ProtocolInfo.proto
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/ProtocolInfo.proto?rev=1496760&r1=1496759&r2=1496760&view=diff
==============================================================================
--- hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/ProtocolInfo.proto (original)
+++ hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/ProtocolInfo.proto Wed Jun 26 06:10:05 2013
@@ -16,6 +16,12 @@
* limitations under the License.
*/
+/**
+ * These .proto interfaces are private and stable.
+ * Please see http://wiki.apache.org/hadoop/Compatibility
+ * for what changes are allowed for a *stable* .proto interface.
+ */
+
option java_package = "org.apache.hadoop.ipc.protobuf";
option java_outer_classname = "ProtocolInfoProtos";
option java_generic_services = true;
Modified: hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto?rev=1496760&r1=1496759&r2=1496760&view=diff
==============================================================================
--- hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto (original)
+++ hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto Wed Jun 26 06:10:05 2013
@@ -15,6 +15,13 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+
+/**
+ * These .proto interfaces are private and stable.
+ * Please see http://wiki.apache.org/hadoop/Compatibility
+ * for what changes are allowed for a *stable* .proto interface.
+ */
+
option java_package = "org.apache.hadoop.ipc.protobuf";
option java_outer_classname = "RpcHeaderProtos";
option java_generate_equals_and_hash = true;
Modified: hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/Security.proto
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/Security.proto?rev=1496760&r1=1496759&r2=1496760&view=diff
==============================================================================
--- hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/Security.proto (original)
+++ hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/Security.proto Wed Jun 26 06:10:05 2013
@@ -16,6 +16,12 @@
* limitations under the License.
*/
+/**
+ * These .proto interfaces are private and stable.
+ * Please see http://wiki.apache.org/hadoop/Compatibility
+ * for what changes are allowed for a *stable* .proto interface.
+ */
+
option java_package = "org.apache.hadoop.security.proto";
option java_outer_classname = "SecurityProtos";
option java_generic_services = true;
Modified: hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/ZKFCProtocol.proto
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/ZKFCProtocol.proto?rev=1496760&r1=1496759&r2=1496760&view=diff
==============================================================================
--- hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/ZKFCProtocol.proto (original)
+++ hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/main/proto/ZKFCProtocol.proto Wed Jun 26 06:10:05 2013
@@ -16,6 +16,12 @@
* limitations under the License.
*/
+/**
+ * These .proto interfaces are private and stable.
+ * Please see http://wiki.apache.org/hadoop/Compatibility
+ * for what changes are allowed for a *stable* .proto interface.
+ */
+
option java_package = "org.apache.hadoop.ha.proto";
option java_outer_classname = "ZKFCProtocolProtos";
option java_generic_services = true;
Modified: hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/site/apt/Compatibility.apt.vm
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/site/apt/Compatibility.apt.vm?rev=1496760&r1=1496759&r2=1496760&view=diff
==============================================================================
--- hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/site/apt/Compatibility.apt.vm (original)
+++ hadoop/common/branches/branch-2.1-beta/hadoop-common-project/hadoop-common/src/site/apt/Compatibility.apt.vm Wed Jun 26 06:10:05 2013
@@ -98,9 +98,8 @@ hand-in-hand to address this.
Wire compatibility concerns data being transmitted over the wire
between Hadoop processes. Hadoop uses Protocol Buffers for most RPC
communication. Preserving compatibility requires prohibiting
- modification to the required fields of the corresponding protocol
- buffer. Optional fields may be added without breaking backwards
- compatibility. Non-RPC communication should be considered as well,
+ modification as described below.
+ Non-RPC communication should be considered as well,
for example using HTTP to transfer an HDFS image as part of
snapshotting or transferring MapTask output. The potential
communications can be categorized as follows:
@@ -131,7 +130,7 @@ hand-in-hand to address this.
* Server-Server compatibility is required to allow mixed versions
within an active cluster so the cluster may be upgraded without
- downtime.
+ downtime in a rolling fashion.
*** Policy
@@ -139,37 +138,56 @@ hand-in-hand to address this.
major release. (Different policies for different categories are yet to be
considered.)
- * The source files generated from the proto files need to be
- compatible within a major release to facilitate rolling
- upgrades. The proto files are governed by the following:
+ * Compatibility can be broken only at a major release, though breaking compatibility
+ even at major releases has grave consequences and should be discussed in the Hadoop community.
- * The following changes are NEVER allowed:
+ * Hadoop protocols are defined in .proto (ProtocolBuffers) files.
+ Client-Server protocols and Server-protocol .proto files are marked as stable.
+ When a .proto file is marked as stable it means that changes should be made
+ in a compatible fashion as described below:
- * Change a field id.
+ * The following changes are compatible and are allowed at any time:
- * Reuse an old field that was previously deleted. Field numbers are
- cheap and changing and reusing is not a good idea.
+ * Add an optional field, with the expectation that the code deals with the field missing due to communication with an older version of the code.
- * The following changes cannot be made to a stable .proto except at a
- major release:
+ * Add a new rpc/method to the service
+
+ * Add a new optional request to a Message
+
+ * Rename a field
+
+ * Rename a .proto file
+
+ * Change .proto annotations that effect code generation (e.g. name of java package)
+
+ * The following changes are incompatible but can be considered only at a major release
+
+ * Change the rpc/method name
+
+ * Change the rpc/method parameter type or return type
+
+ * Remove an rpc/method
+
+ * Change the service name
+
+ * Change the name of a Message
* Modify a field type in an incompatible way (as defined recursively)
- * Add or delete a required field
+ * Change an optional field to required
+
+ * Add or delete a required field
+
+ * Delete an optional field as long as the optional field has reasonable defaults to allow deletions
- * Delete an optional field
+ * The following changes are incompatible and hence never allowed
- * The following changes are allowed at any time:
-
- * Add an optional field, but ensure the code allows communication with prior
- version of the client code which did not have that field.
+ * Change a field id
- * Rename a field
+ * Reuse an old field that was previously deleted.
- * Rename a .proto file
+ * Field numbers are cheap and changing and reusing is not a good idea.
- * Change .proto annotations that effect code generation (e.g. name of
- java package)
** Java Binary compatibility for end-user applications i.e. Apache Hadoop ABI