You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-commits@hadoop.apache.org by at...@apache.org on 2012/02/28 03:21:46 UTC
svn commit: r1294445 - in
/hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs: ./
CHANGES.txt src/main/java/ src/main/native/ src/main/webapps/datanode/
src/main/webapps/hdfs/ src/main/webapps/secondary/ src/test/hdfs/
Author: atm
Date: Tue Feb 28 02:21:42 2012
New Revision: 1294445
URL: http://svn.apache.org/viewvc?rev=1294445&view=rev
Log:
Merge trunk into HA branch.
Modified:
hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/ (props changed)
hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/ (props changed)
hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/native/ (props changed)
hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode/ (props changed)
hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/ (props changed)
hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/ (props changed)
hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs/ (props changed)
Propchange: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Feb 28 02:21:42 2012
@@ -1,4 +1,4 @@
-/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs:1152502-1294248
+/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs:1152502-1294443
/hadoop/core/branches/branch-0.19/hdfs:713112
/hadoop/hdfs/branches/HDFS-1052:987665-1095512
/hadoop/hdfs/branches/HDFS-265:796829-820463
Modified: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt?rev=1294445&r1=1294444&r2=1294445&view=diff
==============================================================================
--- hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt (original)
+++ hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Tue Feb 28 02:21:42 2012
@@ -3,93 +3,24 @@ Hadoop HDFS Change Log
Trunk (unreleased changes)
INCOMPATIBLE CHANGES
-
- HDFS-2676. Remove Avro RPC. (suresh)
NEW FEATURES
-
- HDFS-395. DFS Scalability: Incremental block reports. (Tomasz Nykiel
- via hairong)
-
- HDFS-2517. Add protobuf service for JounralProtocol. (suresh)
-
- HDFS-2518. Add protobuf service for NamenodeProtocol. (suresh)
-
- HDFS-2520. Add protobuf service for InterDatanodeProtocol. (suresh)
-
- HDFS-2519. Add protobuf service for DatanodeProtocol. (suresh)
-
- HDFS-2581. Implement protobuf service for JournalProtocol. (suresh)
-
- HDFS-2618. Implement protobuf service for NamenodeProtocol. (suresh)
-
- HDFS-2629. Implement protobuf service for InterDatanodeProtocol. (suresh)
-
- HDFS-2636. Implement protobuf service for ClientDatanodeProtocol. (suresh)
HDFS-2430. The number of failed or low-resource volumes the NN can tolerate
should be configurable. (atm)
- HDFS-2642. Protobuf translators for DatanodeProtocol. (jitendra)
-
- HDFS-2647. Used protobuf based RPC for InterDatanodeProtocol,
- ClientDatanodeProtocol, JournalProtocol, NamenodeProtocol. (suresh)
-
- HDFS-2666. Fix TestBackupNode failure. (suresh)
-
HDFS-234. Integration with BookKeeper logging system. (Ivan Kelly
via jitendra)
- HDFS-2663. Optional protobuf parameters are not handled correctly.
- (suresh)
-
- HDFS-2661. Enable protobuf RPC for DatanodeProtocol. (jitendra)
-
- HDFS-2697. Move RefreshAuthPolicy, RefreshUserMappings, GetUserMappings
- protocol to protocol buffers. (jitendra)
-
- HDFS-2880. Protobuf changes in DatanodeProtocol to add multiple storages.
- (suresh)
-
- HDFS-2899. Service protocol changes in DatanodeProtocol to add multiple
- storages. (suresh)
-
IMPROVEMENTS
- HADOOP-7524 Change RPC to allow multiple protocols including multuple
- versions of the same protocol (Sanjay Radia)
-
HDFS-1620. Rename HdfsConstants -> HdfsServerConstants, FSConstants ->
HdfsConstants. (Harsh J Chouraria via atm)
- HDFS-2197. Refactor RPC call implementations out of NameNode class (todd)
-
- HDFS-2018. Move all journal stream management code into one place.
- (Ivan Kelly via jitendra)
-
- HDFS-2223. Untangle depencencies between NN components (todd)
- HDFS-2337. DFSClient shouldn't keep multiple RPC proxy references (atm)
-
- HDFS-2351 Change Namenode and Datanode to register each of their protocols
- seperately. (Sanjay Radia)
+ HDFS-2197. Refactor RPC call implementations out of NameNode class (todd)
HDFS-2158. Add JournalSet to manage the set of journals. (jitendra)
- HDFS-2459. Separate datatypes for JournalProtocol. (suresh)
-
- HDFS-2480. Separate datatypes for NamenodeProtocol. (suresh)
-
- HDFS-2181 Separate HDFS Client wire protocol data types (sanjay)
-
- HDFS-2489. Move Finalize and Register to separate file out of
- DatanodeCommand.java. (suresh)
-
- HDFS-2488. Separate datatypes for InterDatanodeProtocol. (suresh)
-
- HDFS-2496. Separate datatypes for DatanodeProtocol. (suresh)
-
- HDFS-2479 HDFS Client Data Types in Protocol Buffers (sanjay)
-
HDFS-2334. Add Closeable to JournalManager. (Ivan Kelly via jitendra)
HDFS-2572. Remove unnecessary double-check in DN#getHostName. (harsh)
@@ -102,30 +33,12 @@ Trunk (unreleased changes)
HDFS-2857. Cleanup BlockInfo class. (suresh)
- HADOOP-7862 Hdfs changes to work with HADOOP 7862:
- Move the support for multiple protocols to lower layer so that Writable,
- PB and Avro can all use it (Sanjay)
-
HDFS-1580. Add interface for generic Write Ahead Logging mechanisms.
(Ivan Kelly via jitendra)
- HDFS-2597 ClientNameNodeProtocol in Protocol Buffers (sanjay)
-
- HDFS-2651 ClientNameNodeProtocol Translators for Protocol Buffers (sanjay)
-
- HDFS-2650. Replace @inheritDoc with @Override. (Hari Mankude via suresh)
-
- HDFS-2669. Enable protobuf rpc for ClientNamenodeProtocol. (Sanjay Radia)
-
- HDFS-2801. Provide a method in client side translators to check for a
- methods supported in underlying protocol. (jitendra)
-
HDFS-208. name node should warn if only one dir is listed in dfs.name.dir.
(Uma Maheswara Rao G via eli)
- HDS-2895. Remove Writable wire protocol types and translators to
- complete transition to protocol buffers. (suresh)
-
HDFS-2786. Fix host-based token incompatibilities in DFSUtil. (Kihwal Lee
via jitendra)
@@ -148,6 +61,7 @@ Trunk (unreleased changes)
(suresh)
OPTIMIZATIONS
+
HDFS-2477. Optimize computing the diff between a block report and the
namenode state. (Tomasz Nykiel via hairong)
@@ -158,6 +72,7 @@ Trunk (unreleased changes)
over-replicated, and invalidated blocks. (Tomasz Nykiel via todd)
BUG FIXES
+
HDFS-2299. TestOfflineEditsViewer is failing on trunk. (Uma Maheswara Rao G
via atm)
HDFS-2310. TestBackupNode fails since HADOOP-7524 went in.
@@ -180,10 +95,118 @@ Trunk (unreleased changes)
HDFS-2188. Make FSEditLog create its journals from a list of URIs rather
than NNStorage. (Ivan Kelly via jitendra)
- HDFS-2481 Unknown protocol: org.apache.hadoop.hdfs.protocol.ClientProtocol.
+ HDFS-1765. Block Replication should respect under-replication
+ block priority. (Uma Maheswara Rao G via eli)
+
+ HDFS-2765. TestNameEditsConfigs is incorrectly swallowing IOE. (atm)
+
+ HDFS-2776. Missing interface annotation on JournalSet.
+ (Brandon Li via jitendra)
+
+ HDFS-2759. Pre-allocate HDFS edit log files after writing version number.
+ (atm)
+
+ HDFS-2908. Add apache license header for StorageReport.java. (Brandon Li
+ via jitendra)
+
+Release 0.23.3 - UNRELEASED
+
+ INCOMPATIBLE CHANGES
+
+ HDFS-2676. Remove Avro RPC. (suresh)
+
+ NEW FEATURES
+
+ HDFS-2978. The NameNode should expose name dir statuses via JMX. (atm)
+
+ HDFS-395. DFS Scalability: Incremental block reports. (Tomasz Nykiel
+ via hairong)
+
+ HDFS-2517. Add protobuf service for JounralProtocol. (suresh)
+
+ HDFS-2518. Add protobuf service for NamenodeProtocol. (suresh)
+
+ HDFS-2520. Add protobuf service for InterDatanodeProtocol. (suresh)
+
+ HDFS-2519. Add protobuf service for DatanodeProtocol. (suresh)
+
+ HDFS-2581. Implement protobuf service for JournalProtocol. (suresh)
+
+ HDFS-2618. Implement protobuf service for NamenodeProtocol. (suresh)
+
+ HDFS-2629. Implement protobuf service for InterDatanodeProtocol. (suresh)
+
+ HDFS-2636. Implement protobuf service for ClientDatanodeProtocol. (suresh)
+
+ HDFS-2642. Protobuf translators for DatanodeProtocol. (jitendra)
+
+ HDFS-2647. Used protobuf based RPC for InterDatanodeProtocol,
+ ClientDatanodeProtocol, JournalProtocol, NamenodeProtocol. (suresh)
+
+ HDFS-2661. Enable protobuf RPC for DatanodeProtocol. (jitendra)
+
+ HDFS-2697. Move RefreshAuthPolicy, RefreshUserMappings, GetUserMappings
+ protocol to protocol buffers. (jitendra)
+
+ HDFS-2880. Protobuf changes in DatanodeProtocol to add multiple storages.
+ (suresh)
+
+ HDFS-2899. Service protocol changes in DatanodeProtocol to add multiple
+ storages. (suresh)
+
+ IMPROVEMENTS
+
+ HDFS-2018. Move all journal stream management code into one place.
+ (Ivan Kelly via jitendra)
+
+ HDFS-2223. Untangle depencencies between NN components (todd)
+
+ HDFS-2351. Change Namenode and Datanode to register each of their protocols
+ seperately (sanjay)
+
+ HDFS-2337. DFSClient shouldn't keep multiple RPC proxy references (atm)
+
+ HDFS-2181. Separate HDFS Client wire protocol data types (sanjay)
+
+ HDFS-2459. Separate datatypes for Journal Protocol. (suresh)
+
+ HDFS-2480. Separate datatypes for NamenodeProtocol. (suresh)
+
+ HDFS-2489. Move Finalize and Register to separate file out of
+ DatanodeCommand.java. (suresh)
+
+ HDFS-2488. Separate datatypes for InterDatanodeProtocol. (suresh)
+
+ HDFS-2496. Separate datatypes for DatanodeProtocol. (suresh)
+
+ HDFS-2479. HDFS Client Data Types in Protocol Buffers (sanjay)
+
+ HADOOP-7862. Hdfs changes to work with HADOOP-7862: Move the support for
+ multiple protocols to lower layer so that Writable, PB and Avro can all
+ use it. (sanjay)
+
+ HDFS-2597. ClientNameNodeProtocol in Protocol Buffers. (sanjay)
+
+ HDFS-2651. ClientNameNodeProtocol Translators for Protocol Buffers. (sanjay)
+
+ HDFS-2650. Replace @inheritDoc with @Override. (Hari Mankude via suresh).
+
+ HDFS-2669. Enable protobuf rpc for ClientNamenodeProtocol. (sanjay)
+
+ HDFS-2801. Provide a method in client side translators to check for a
+ methods supported in underlying protocol. (jitendra)
+
+ HDFS-2895. Remove Writable wire protocol types and translators to
+ complete transition to protocol buffers. (suresh)
+
+ OPTIMIZATIONS
+
+ BUG FIXES
+
+ HDFS-2481. Unknown protocol: org.apache.hadoop.hdfs.protocol.ClientProtocol.
(sanjay)
- HDFS-2497 Fix TestBackupNode failure. (suresh)
+ HDFS-2497. Fix TestBackupNode failure. (suresh)
HDFS-2499. RPC client is created incorrectly introduced in HDFS-2459.
(suresh)
@@ -194,8 +217,9 @@ Trunk (unreleased changes)
HDFS-2532. TestDfsOverAvroRpc timing out in trunk (Uma Maheswara Rao G
via todd)
- HDFS-1765. Block Replication should respect under-replication
- block priority. (Uma Maheswara Rao G via eli)
+ HDFS-2666. Fix TestBackupNode failure. (suresh)
+
+ HDFS-2663. Optional protobuf parameters are not handled correctly. (suresh)
HDFS-2694. Removal of Avro broke non-PB NN services. (atm)
@@ -205,39 +229,14 @@ Trunk (unreleased changes)
HDFS-2700. Fix failing TestDataNodeMultipleRegistrations in trunk
(Uma Maheswara Rao G via todd)
- HDFS-2765. TestNameEditsConfigs is incorrectly swallowing IOE. (atm)
-
HDFS-2739. SecondaryNameNode doesn't start up. (jitendra)
- HDFS-2776. Missing interface annotation on JournalSet.
- (Brandon Li via jitendra)
-
HDFS-2768. BackupNode stop can not close proxy connections because
it is not a proxy instance. (Uma Maheswara Rao G via eli)
- HDFS-2759. Pre-allocate HDFS edit log files after writing version number.
- (atm)
-
- HDFS-2908. Add apache license header for StorageReport.java. (Brandon Li
- via jitendra)
-
HDFS-2968. Protocol translator for BlockRecoveryCommand broken when
multiple blocks need recovery. (todd)
-Release 0.23.3 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- HDFS-2978. The NameNode should expose name dir statuses via JMX. (atm)
-
- IMPROVEMENTS
-
- OPTIMIZATIONS
-
- BUG FIXES
-
Release 0.23.2 - UNRELEASED
INCOMPATIBLE CHANGES
Propchange: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Feb 28 02:21:42 2012
@@ -1,4 +1,4 @@
-/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java:1159757-1294248
+/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java:1159757-1294443
/hadoop/core/branches/branch-0.19/hdfs/src/java:713112
/hadoop/core/branches/branch-0.19/hdfs/src/main/java:713112
/hadoop/core/trunk/src/hdfs:776175-785643,785929-786278
Propchange: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/native/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Feb 28 02:21:42 2012
@@ -1,4 +1,4 @@
-/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native:1152502-1294248
+/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native:1152502-1294443
/hadoop/core/branches/branch-0.19/hdfs/src/main/native:713112
/hadoop/core/branches/branch-0.19/mapred/src/c++/libhdfs:713112
/hadoop/core/trunk/src/c++/libhdfs:776175-784663
Propchange: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Feb 28 02:21:42 2012
@@ -1,4 +1,4 @@
-/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode:1159757-1294248
+/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode:1159757-1294443
/hadoop/core/branches/branch-0.19/hdfs/src/main/webapps/datanode:713112
/hadoop/core/branches/branch-0.19/hdfs/src/webapps/datanode:713112
/hadoop/core/trunk/src/webapps/datanode:776175-784663
Propchange: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Feb 28 02:21:42 2012
@@ -1,4 +1,4 @@
-/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs:1152502-1294248
+/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs:1152502-1294443
/hadoop/core/branches/branch-0.19/hdfs/src/main/webapps/hdfs:713112
/hadoop/core/branches/branch-0.19/hdfs/src/webapps/hdfs:713112
/hadoop/core/trunk/src/webapps/hdfs:776175-784663
Propchange: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Feb 28 02:21:42 2012
@@ -1,4 +1,4 @@
-/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary:1152502-1294248
+/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary:1152502-1294443
/hadoop/core/branches/branch-0.19/hdfs/src/main/webapps/secondary:713112
/hadoop/core/branches/branch-0.19/hdfs/src/webapps/secondary:713112
/hadoop/core/trunk/src/webapps/secondary:776175-784663
Propchange: hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Feb 28 02:21:42 2012
@@ -1,4 +1,4 @@
-/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs:1159757-1294248
+/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs:1159757-1294443
/hadoop/core/branches/branch-0.19/hdfs/src/test/hdfs:713112
/hadoop/core/trunk/src/test/hdfs:776175-785643
/hadoop/hdfs/branches/HDFS-1052/src/test/hdfs:987665-1095512