You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-commits@hadoop.apache.org by st...@apache.org on 2009/08/26 17:03:35 UTC
svn commit: r808037 [2/3] - in /hadoop/hdfs/branches/HDFS-326: ./ ivy/ lib/
src/contrib/hdfsproxy/ src/java/ src/java/org/apache/hadoop/hdfs/
src/java/org/apache/hadoop/hdfs/protocol/
src/java/org/apache/hadoop/hdfs/server/common/ src/java/org/apache/h...
Modified: hadoop/hdfs/branches/HDFS-326/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/hdfs/branches/HDFS-326/CHANGES.txt?rev=808037&r1=808036&r2=808037&view=diff
==============================================================================
--- hadoop/hdfs/branches/HDFS-326/CHANGES.txt (original)
+++ hadoop/hdfs/branches/HDFS-326/CHANGES.txt Wed Aug 26 15:03:33 2009
@@ -2,6 +2,12 @@
Trunk (unreleased changes)
+ INCOMPATIBLE CHANGES
+
+ HDFS-538. Per the contract elucidated in HADOOP-6201, throw
+ FileNotFoundException from FileSystem::listStatus rather than returning
+ null. (Jakob Homan via cdouglas)
+
NEW FEATURES
HDFS-436. Introduce AspectJ framework for HDFS code and tests.
@@ -47,7 +53,7 @@
via szetszwo)
HDFS-493. Change build.xml so that the fault-injected tests are executed
- only by the run-test-*-faul-inject targets. (Konstantin Boudnik via
+ only by the run-test-*-fault-inject targets. (Konstantin Boudnik via
szetszwo)
HDFS-446. Improvements to Offline Image Viewer. (Jakob Homan via shv)
@@ -81,21 +87,33 @@
HDFS-524. Further DataTransferProtocol code refactoring. (szetszwo)
- HDFS-527. Remove/deprecate unnecessary DFSClient constructors. (szetszwo)
-
HDFS-529. Use BlockInfo instead of Block to avoid redundant block searches
in BlockManager. (shv)
HDFS-530. Refactor TestFileAppend* to remove code duplication.
(Konstantin Boudnik via szetszwo)
- HDFS-451. Add fault injection tests, Pipeline_Fi_06,07,14,15, for
- DataTransferProtocol. (szetszwo)
+ HDFS-451. Add fault injection tests for DataTransferProtocol. (szetszwo)
HDFS-409. Add more access token tests. (Kan Zhang via szetszwo)
HDFS-546. DatanodeDescriptor iterator blocks as BlockInfo. (shv)
+ HDFS-457. Do not shutdown datanode if some, but not all, volumes fail.
+ (Boris Shkolnik via szetszwo)
+
+ HDFS-548. TestFsck takes nearly 10 minutes to run. (hairong)
+
+ HDFS-539. Refactor fault injeciton pipeline test util for future reuse.
+ (Konstantin Boudnik via szetszwo)
+
+ HDFS-552. Change TestFiDataTransferProtocol to junit 4 and add a few new
+ tests. (szetszwo)
+
+ HDFS-549. Allow a non-fault-inject test, which is specified by -Dtestcase,
+ to be executed by the run-test-hdfs-fault-inject target. (Konstantin
+ Boudnik via szetszwo)
+
BUG FIXES
HDFS-76. Better error message to users when commands fail because of
@@ -108,7 +126,7 @@
HADOOP-6096. Fix Eclipse project and classpath files following project
split. (tomwhite)
- HDFS-195. Handle expired tokens when write pipeline is restablished.
+ HDFS-195. Handle expired tokens when write pipeline is reestablished.
(Kan Zhang via rangadi)
HDFS-181. Validate src path in FSNamesystem.getFileInfo(..). (Todd
@@ -147,14 +165,17 @@
HDFS-119. Fix a bug in logSync(), which causes NameNode block forever.
(Suresh Srinivas via shv)
- HDFS-167. Fix a bug in DFSClient that caused infinite retries on write.
- (Bill Zeller via szetszwo)
-
HDFS-534. Include avro in ivy. (szetszwo)
HDFS-532. Allow applications to know that a read request failed
because block is missing. (dhruba)
+ HDFS-561. Fix write pipeline READ_TIMEOUT in DataTransferProtocol.
+ (Kan Zhang via szetszwo)
+
+ HDFS-553. BlockSender reports wrong failed position in ChecksumException.
+ (hairong)
+
Release 0.20.1 - Unreleased
IMPROVEMENTS
@@ -165,3 +186,46333 @@
HDFS-525. The SimpleDateFormat object in ListPathsServlet is not thread
safe. (Suresh Srinivas via szetszwo)
+<<<<<<< .working
+=======
+Hadoop Change Log
+
+Trunk (unreleased changes)
+
+ INCOMPATIBLE CHANGES
+
+ HADOOP-4895. Remove deprecated methods DFSClient.getHints(..) and
+ DFSClient.isDirectory(..). (szetszwo)
+
+ HADOOP-4941. Remove deprecated FileSystem methods: getBlockSize(Path f),
+ getLength(Path f) and getReplication(Path src). (szetszwo)
+
+ HADOOP-4648. Remove obsolete, deprecated InMemoryFileSystem and
+ ChecksumDistributedFileSystem. (cdouglas via szetszwo)
+
+ HADOOP-4940. Remove a deprecated method FileSystem.delete(Path f). (Enis
+ Soztutar via szetszwo)
+
+ HADOOP-4010. Change semantics for LineRecordReader to read an additional
+ line per split- rather than moving back one character in the stream- to
+ work with splittable compression codecs. (Abdul Qadeer via cdouglas)
+
+ HADOOP-5094. Show hostname and separate live/dead datanodes in DFSAdmin
+ report. (Jakob Homan via szetszwo)
+
+ HADOOP-4942. Remove deprecated FileSystem methods getName() and
+ getNamed(String name, Configuration conf). (Jakob Homan via szetszwo)
+
+ HADOOP-5486. Removes the CLASSPATH string from the command line and instead
+ exports it in the environment. (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-2827. Remove deprecated NetUtils::getServerAddress. (cdouglas)
+
+ HADOOP-5681. Change examples RandomWriter and RandomTextWriter to
+ use new mapreduce API. (Amareshwari Sriramadasu via sharad)
+
+ HADOOP-5680. Change org.apache.hadoop.examples.SleepJob to use new
+ mapreduce api. (Amareshwari Sriramadasu via sharad)
+
+ HADOOP-5699. Change org.apache.hadoop.examples.PiEstimator to use
+ new mapreduce api. (Amareshwari Sriramadasu via sharad)
+
+ HADOOP-5720. Introduces new task types - JOB_SETUP, JOB_CLEANUP
+ and TASK_CLEANUP. Removes the isMap methods from TaskID/TaskAttemptID
+ classes. (ddas)
+
+ HADOOP-5668. Change TotalOrderPartitioner to use new API. (Amareshwari
+ Sriramadasu via cdouglas)
+
+ HADOOP-5738. Split "waiting_tasks" JobTracker metric into waiting maps and
+ waiting reduces. (Sreekanth Ramakrishnan via cdouglas)
+
+ HADOOP-5679. Resolve findbugs warnings in core/streaming/pipes/examples.
+ (Jothi Padmanabhan via sharad)
+
+ HADOOP-4359. Support for data access authorization checking on Datanodes.
+ (Kan Zhang via rangadi)
+
+ HADOOP-5690. Change org.apache.hadoop.examples.DBCountPageView to use
+ new mapreduce api. (Amareshwari Sriramadasu via sharad)
+
+ HADOOP-5694. Change org.apache.hadoop.examples.dancing to use new
+ mapreduce api. (Amareshwari Sriramadasu via sharad)
+
+ HADOOP-5696. Change org.apache.hadoop.examples.Sort to use new
+ mapreduce api. (Amareshwari Sriramadasu via sharad)
+
+ HADOOP-5698. Change org.apache.hadoop.examples.MultiFileWordCount to
+ use new mapreduce api. (Amareshwari Sriramadasu via sharad)
+
+ HADOOP-5913. Provide ability to an administrator to stop and start
+ job queues. (Rahul Kumar Singh and Hemanth Yamijala via yhemanth)
+
+ NEW FEATURES
+
+ HADOOP-4268. Change fsck to use ClientProtocol methods so that the
+ corresponding permission requirement for running the ClientProtocol
+ methods will be enforced. (szetszwo)
+
+ HADOOP-3953. Implement sticky bit for directories in HDFS. (Jakob Homan
+ via szetszwo)
+
+ HADOOP-4368. Implement df in FsShell to show the status of a FileSystem.
+ (Craig Macdonald via szetszwo)
+
+ HADOOP-3741. Add a web ui to the SecondaryNameNode for showing its status.
+ (szetszwo)
+
+ HADOOP-5018. Add pipelined writers to Chukwa. (Ari Rabkin via cdouglas)
+
+ HADOOP-5052. Add an example computing exact digits of pi using the
+ Bailey-Borwein-Plouffe algorithm. (Tsz Wo (Nicholas), SZE via cdouglas)
+
+ HADOOP-4927. Adds a generic wrapper around outputformat to allow creation of
+ output on demand (Jothi Padmanabhan via ddas)
+
+ HADOOP-5144. Add a new DFSAdmin command for changing the setting of restore
+ failed storage replicas in namenode. (Boris Shkolnik via szetszwo)
+
+ HADOOP-5258. Add a new DFSAdmin command to print a tree of the rack and
+ datanode topology as seen by the namenode. (Jakob Homan via szetszwo)
+
+ HADOOP-4756. A command line tool to access JMX properties on NameNode
+ and DataNode. (Boris Shkolnik via rangadi)
+
+ HADOOP-4539. Introduce backup node and checkpoint node. (shv)
+
+ HADOOP-5363. Add support for proxying connections to multiple clusters with
+ different versions to hdfsproxy. (Zhiyong Zhang via cdouglas)
+
+ HADOOP-5528. Add a configurable hash partitioner operating on ranges of
+ BinaryComparable keys. (Klaas Bosteels via shv)
+
+ HADOOP-5257. HDFS servers may start and stop external components through
+ a plugin interface. (Carlos Valiente via dhruba)
+
+ HADOOP-5450. Add application-specific data types to streaming's typed bytes
+ interface. (Klaas Bosteels via omalley)
+
+ HADOOP-5518. Add contrib/mrunit, a MapReduce unit test framework.
+ (Aaron Kimball via cutting)
+
+ HADOOP-5469. Add /metrics servlet to daemons, providing metrics
+ over HTTP as either text or JSON. (Philip Zeyliger via cutting)
+
+ HADOOP-5467. Introduce offline fsimage image viewer. (Jakob Homan via shv)
+
+ HADOOP-5752. Add a new hdfs image processor, Delimited, to oiv. (Jakob
+ Homan via szetszwo)
+
+ HADOOP-5266. Adds the capability to do mark/reset of the reduce values
+ iterator in the Context object API. (Jothi Padmanabhan via ddas)
+
+ HADOOP-5745. Allow setting the default value of maxRunningJobs for all
+ pools. (dhruba via matei)
+
+ HADOOP-5643. Adds a way to decommission TaskTrackers while the JobTracker
+ is running. (Amar Kamat via ddas)
+
+ HADOOP-4829. Allow FileSystem shutdown hook to be disabled.
+ (Todd Lipcon via tomwhite)
+
+ HADOOP-5815. Sqoop: A database import tool for Hadoop.
+ (Aaron Kimball via tomwhite)
+
+ HADOOP-4861. Add disk usage with human-readable size (-duh).
+ (Todd Lipcon via tomwhite)
+
+ HADOOP-5844. Use mysqldump when connecting to local mysql instance in Sqoop.
+ (Aaron Kimball via tomwhite)
+
+ HADOOP-5170. Allows jobs to set max maps/reduces per-node and per-cluster.
+ (Matei Zaharia via ddas)
+
+ HADOOP-5897. Add name-node metrics to capture java heap usage.
+ (Suresh Srinivas via shv)
+
+ IMPROVEMENTS
+
+ HADOOP-4565. Added CombineFileInputFormat to use data locality information
+ to create splits. (dhruba via zshao)
+
+ HADOOP-4936. Improvements to TestSafeMode. (shv)
+
+ HADOOP-4985. Remove unnecessary "throw IOException" declarations in
+ FSDirectory related methods. (szetszwo)
+
+ HADOOP-5017. Change NameNode.namesystem declaration to private. (szetszwo)
+
+ HADOOP-4794. Add branch information from the source version control into
+ the version information that is compiled into Hadoop. (cdouglas via
+ omalley)
+
+ HADOOP-5070. Increment copyright year to 2009, remove assertions of ASF
+ copyright to licensed files. (Tsz Wo (Nicholas), SZE via cdouglas)
+
+ HADOOP-5037. Deprecate static FSNamesystem.getFSNamesystem(). (szetszwo)
+
+ HADOOP-5088. Include releaseaudit target as part of developer test-patch
+ target. (Giridharan Kesavan via nigel)
+
+ HADOOP-2721. Uses setsid when creating new tasks so that subprocesses of
+ this process will be within this new session (and this process will be
+ the process leader for all the subprocesses). Killing the process leader,
+ or the main Java task in Hadoop's case, kills the entire subtree of
+ processes. (Ravi Gummadi via ddas)
+
+ HADOOP-5097. Remove static variable JspHelper.fsn, a static reference to
+ a non-singleton FSNamesystem object. (szetszwo)
+
+ HADOOP-3327. Improves handling of READ_TIMEOUT during map output copying.
+ (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-5124. Choose datanodes randomly instead of starting from the first
+ datanode for providing fairness. (hairong via szetszwo)
+
+ HADOOP-4930. Implement a Linux native executable that can be used to
+ launch tasks as users. (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-5122. Fix format of fs.default.name value in libhdfs test conf.
+ (Craig Macdonald via tomwhite)
+
+ HADOOP-5038. Direct daemon trace to debug log instead of stdout. (Jerome
+ Boulon via cdouglas)
+
+ HADOOP-5101. Improve packaging by adding 'all-jars' target building core,
+ tools, and example jars. Let findbugs depend on this rather than the 'tar'
+ target. (Giridharan Kesavan via cdouglas)
+
+ HADOOP-4868. Splits the hadoop script into three parts - bin/hadoop,
+ bin/mapred and bin/hdfs. (Sharad Agarwal via ddas)
+
+ HADOOP-1722. Adds support for TypedBytes and RawBytes in Streaming.
+ (Klaas Bosteels via ddas)
+
+ HADOOP-4220. Changes the JobTracker restart tests so that they take much
+ less time. (Amar Kamat via ddas)
+
+ HADOOP-4885. Try to restore failed name-node storage directories at
+ checkpoint time. (Boris Shkolnik via shv)
+
+ HADOOP-5209. Update year to 2009 for javadoc. (szetszwo)
+
+ HADOOP-5279. Remove unnecessary targets from test-patch.sh.
+ (Giridharan Kesavan via nigel)
+
+ HADOOP-5120. Remove the use of FSNamesystem.getFSNamesystem() from
+ UpgradeManagerNamenode and UpgradeObjectNamenode. (szetszwo)
+
+ HADOOP-5222. Add offset to datanode clienttrace. (Lei Xu via cdouglas)
+
+ HADOOP-5240. Skip re-building javadoc when it is already
+ up-to-date. (Aaron Kimball via cutting)
+
+ HADOOP-5042. Add a cleanup stage to log rollover in Chukwa appender.
+ (Jerome Boulon via cdouglas)
+
+ HADOOP-5264. Removes redundant configuration object from the TaskTracker.
+ (Sharad Agarwal via ddas)
+
+ HADOOP-5232. Enable patch testing to occur on more than one host.
+ (Giri Kesavan via nigel)
+
+ HADOOP-4546. Fix DF reporting for AIX. (Bill Habermaas via cdouglas)
+
+ HADOOP-5023. Add Tomcat support to HdfsProxy. (Zhiyong Zhang via cdouglas)
+
+ HADOOP-5317. Provide documentation for LazyOutput Feature.
+ (Jothi Padmanabhan via johan)
+
+ HADOOP-5455. Document rpc metrics context to the extent dfs, mapred, and
+ jvm contexts are documented. (Philip Zeyliger via cdouglas)
+
+ HADOOP-5358. Provide scripting functionality to the synthetic load
+ generator. (Jakob Homan via hairong)
+
+ HADOOP-5442. Paginate jobhistory display and added some search
+ capabilities. (Amar Kamat via acmurthy)
+
+ HADOOP-4842. Streaming now allows specifiying a command for the combiner.
+ (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-5196. avoiding unnecessary byte[] allocation in
+ SequenceFile.CompressedBytes and SequenceFile.UncompressedBytes.
+ (hong tang via mahadev)
+
+ HADOOP-4655. New method FileSystem.newInstance() that always returns
+ a newly allocated FileSystem object. (dhruba)
+
+ HADOOP-4788. Set Fair scheduler to assign both a map and a reduce on each
+ heartbeat by default. (matei)
+
+ HADOOP-5491. In contrib/index, better control memory usage.
+ (Ning Li via cutting)
+
+ HADOOP-5423. Include option of preserving file metadata in
+ SequenceFile::sort. (Michael Tamm via cdouglas)
+
+ HADOOP-5331. Add support for KFS appends. (Sriram Rao via cdouglas)
+
+ HADOOP-4365. Make Configuration::getProps protected in support of
+ meaningful subclassing. (Steve Loughran via cdouglas)
+
+ HADOOP-2413. Remove the static variable FSNamesystem.fsNamesystemObject.
+ (Konstantin Shvachko via szetszwo)
+
+ HADOOP-4584. Improve datanode block reports and associated file system
+ scan to avoid interefering with normal datanode operations.
+ (Suresh Srinivas via rangadi)
+
+ HADOOP-5502. Documentation for backup and checkpoint nodes.
+ (Jakob Homan via shv)
+
+ HADOOP-5485. Mask actions in the fair scheduler's servlet UI based on
+ value of webinterface.private.actions.
+ (Vinod Kumar Vavilapalli via yhemanth)
+
+ HADOOP-5581. HDFS should throw FileNotFoundException when while opening
+ a file that does not exist. (Brian Bockelman via rangadi)
+
+ HADOOP-5509. PendingReplicationBlocks does not start monitor in the
+ constructor. (shv)
+
+ HADOOP-5494. Modify sorted map output merger to lazily read values,
+ rather than buffering at least one record for each segment. (Devaraj Das
+ via cdouglas)
+
+ HADOOP-5396. Provide ability to refresh queue ACLs in the JobTracker
+ without having to restart the daemon.
+ (Sreekanth Ramakrishnan and Vinod Kumar Vavilapalli via yhemanth)
+
+ HADOOP-4490. Provide ability to run tasks as job owners.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-5697. Change org.apache.hadoop.examples.Grep to use new
+ mapreduce api. (Amareshwari Sriramadasu via sharad)
+
+ HADOOP-5625. Add operation duration to clienttrace. (Lei Xu via cdouglas)
+
+ HADOOP-5705. Improve TotalOrderPartitioner efficiency by updating the trie
+ construction. (Dick King via cdouglas)
+
+ HADOOP-5589. Eliminate source limit of 64 for map-side joins imposed by
+ TupleWritable encoding. (Jingkei Ly via cdouglas)
+
+ HADOOP-5734. Correct block placement policy description in HDFS
+ Design document. (Konstantin Boudnik via shv)
+
+ HADOOP-5657. Validate data in TestReduceFetch to improve merge test
+ coverage. (cdouglas)
+
+ HADOOP-5613. Change S3Exception to checked exception.
+ (Andrew Hitchcock via tomwhite)
+
+ HADOOP-5717. Create public enum class for the Framework counters in
+ org.apache.hadoop.mapreduce. (Amareshwari Sriramadasu via sharad)
+
+ HADOOP-5217. Split AllTestDriver for core, hdfs and mapred. (sharad)
+
+ HADOOP-5364. Add certificate expiration warning to HsftpFileSystem and HDFS
+ proxy. (Zhiyong Zhang via cdouglas)
+
+ HADOOP-5733. Add map/reduce slot capacity and blacklisted capacity to
+ JobTracker metrics. (Sreekanth Ramakrishnan via cdouglas)
+
+ HADOOP-5596. Add EnumSetWritable. (He Yongqiang via szetszwo)
+
+ HADOOP-5727. Simplify hashcode for ID types. (Shevek via cdouglas)
+
+ HADOOP-5500. In DBOutputFormat, where field names are absent permit the
+ number of fields to be sufficient to construct the select query. (Enis
+ Soztutar via cdouglas)
+
+ HADOOP-5081. Split TestCLI into HDFS, Mapred and Core tests. (sharad)
+
+ HADOOP-5015. Separate block management code from FSNamesystem. (Suresh
+ Srinivas via szetszwo)
+
+ HADOOP-5080. Add new test cases to TestMRCLI and TestHDFSCLI
+ (V.Karthikeyan via nigel)
+
+ HADOOP-5135. Splits the tests into different directories based on the
+ package. Four new test targets have been defined - run-test-core,
+ run-test-mapred, run-test-hdfs and run-test-hdfs-with-mr.
+ (Sharad Agarwal via ddas)
+
+ HADOOP-5771. Implements unit tests for LinuxTaskController.
+ (Sreekanth Ramakrishnan and Vinod Kumar Vavilapalli via yhemanth)
+
+ HADOOP-5419. Provide a facility to query the Queue ACLs for the
+ current user.
+ (Rahul Kumar Singh via yhemanth)
+
+ HADOOP-5780. Improve per block message prited by "-metaSave" in HDFS.
+ (Raghu Angadi)
+
+ HADOOP-5823. Added a new class DeprecatedUTF8 to help with removing
+ UTF8 related javac warnings. These warnings are removed in
+ FSEditLog.java as a use case. (Raghu Angadi)
+
+ HADOOP-5824. Deprecate DataTransferProtocol.OP_READ_METADATA and remove
+ the corresponding unused codes. (Kan Zhang via szetszwo)
+
+ HADOOP-5721. Factor out EditLogFileInputStream and EditLogFileOutputStream
+ into independent classes. (Luca Telloli & Flavio Junqueira via shv)
+
+ HADOOP-5838. Fix a few javac warnings in HDFS. (Raghu Angadi)
+
+ HADOOP-5854. Fix a few "Inconsistent Synchronization" warnings in HDFS.
+ (Raghu Angadi)
+
+ HADOOP-5369. Small tweaks to reduce MapFile index size. (Ben Maurer
+ via sharad)
+
+ HADOOP-5858. Eliminate UTF8 and fix warnings in test/hdfs-with-mr package.
+ (shv)
+
+ HADOOP-5866. Move DeprecatedUTF8 from o.a.h.io to o.a.h.hdfs since it may
+ not be used outside hdfs. (Raghu Angadi)
+
+ HADOOP-5857. Move normal java methods from hdfs .jsp files to .java files.
+ (szetszwo)
+
+ HADOOP-5873. Remove deprecated methods randomDataNode() and
+ getDatanodeByIndex(..) in FSNamesystem. (szetszwo)
+
+ HADOOP-5572. Improves the progress reporting for the sort phase for both
+ maps and reduces. (Ravi Gummadi via ddas)
+
+ HADOOP-5839. Fix EC2 scripts to allow remote job submission.
+ (Joydeep Sen Sarma via tomwhite)
+
+ HADOOP-5877. Fix javac warnings in TestHDFSServerPorts, TestCheckpoint,
+ TestNameEditsConfig, TestStartup and TestStorageRestore.
+ (Jakob Homan via shv)
+
+ HADOOP-5438. Provide a single FileSystem method to create or open-for-append
+ to a file. (He Yongqiang via dhruba)
+
+ HADOOP-5472. Change DistCp to support globbing of input paths. (Dhruba
+ Borthakur and Rodrigo Schmidt via szetszwo)
+
+ HADOOP-5175. Don't unpack libjars on classpath. (Todd Lipcon via tomwhite)
+
+ HADOOP-5620. Add an option to DistCp for preserving modification and access
+ times. (Rodrigo Schmidt via szetszwo)
+
+ HADOOP-5664. Change map serialization so a lock is obtained only where
+ contention is possible, rather than for each write. (cdouglas)
+
+ HADOOP-5896. Remove the dependency of GenericOptionsParser on
+ Option.withArgPattern. (Giridharan Kesavan and Sharad Agarwal via
+ sharad)
+
+ HADOOP-5784. Makes the number of heartbeats that should arrive a second
+ at the JobTracker configurable. (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-5955. Changes TestFileOuputFormat so that is uses LOCAL_MR
+ instead of CLUSTER_MR. (Jothi Padmanabhan via das)
+
+ HADOOP-5948. Changes TestJavaSerialization to use LocalJobRunner
+ instead of MiniMR/DFS cluster. (Jothi Padmanabhan via das)
+
+ HADOOP-2838. Add mapred.child.env to pass environment variables to
+ tasktracker's child processes. (Amar Kamat via sharad)
+
+ HADOOP-5961. DataNode process understand generic hadoop command line
+ options (like -Ddfs.property=value). (Raghu Angadi)
+
+ HADOOP-5938. Change org.apache.hadoop.mapred.jobcontrol to use new
+ api. (Amareshwari Sriramadasu via sharad)
+
+ HADOOP-2141. Improves the speculative execution heuristic. The heuristic
+ is currently based on the progress-rates of tasks and the expected time
+ to complete. Also, statistics about trackers are collected, and speculative
+ tasks are not given to the ones deduced to be slow.
+ (Andy Konwinski and ddas)
+
+ OPTIMIZATIONS
+
+ HADOOP-5595. NameNode does not need to run a replicator to choose a
+ random DataNode. (hairong)
+
+ HADOOP-5603. Improve NameNode's block placement performance. (hairong)
+
+ HADOOP-5638. More improvement on block placement performance. (hairong)
+
+ BUG FIXES
+
+ HADOOP-5379. CBZip2InputStream to throw IOException on data crc error.
+ (Rodrigo Schmidt via zshao)
+
+ HADOOP-5326. Fixes CBZip2OutputStream data corruption problem.
+ (Rodrigo Schmidt via zshao)
+
+ HADOOP-4963. Fixes a logging to do with getting the location of
+ map output file. (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-2337. Trash should close FileSystem on exit and should not start
+ emtying thread if disabled. (shv)
+
+ HADOOP-5072. Fix failure in TestCodec because testSequenceFileGzipCodec
+ won't pass without native gzip codec. (Zheng Shao via dhruba)
+
+ HADOOP-5050. TestDFSShell.testFilePermissions should not assume umask
+ setting. (Jakob Homan via szetszwo)
+
+ HADOOP-4975. Set classloader for nested mapred.join configs. (Jingkei Ly
+ via cdouglas)
+
+ HADOOP-5078. Remove invalid AMI kernel in EC2 scripts. (tomwhite)
+
+ HADOOP-5045. FileSystem.isDirectory() should not be deprecated. (Suresh
+ Srinivas via szetszwo)
+
+ HADOOP-4960. Use datasource time, rather than system time, during metrics
+ demux. (Eric Yang via cdouglas)
+
+ HADOOP-5032. Export conf dir set in config script. (Eric Yang via cdouglas)
+
+ HADOOP-5176. Fix a typo in TestDFSIO. (Ravi Phulari via szetszwo)
+
+ HADOOP-4859. Distinguish daily rolling output dir by adding a timestamp.
+ (Jerome Boulon via cdouglas)
+
+ HADOOP-4959. Correct system metric collection from top on Redhat 5.1. (Eric
+ Yang via cdouglas)
+
+ HADOOP-5039. Fix log rolling regex to process only the relevant
+ subdirectories. (Jerome Boulon via cdouglas)
+
+ HADOOP-5095. Update Chukwa watchdog to accept config parameter. (Jerome
+ Boulon via cdouglas)
+
+ HADOOP-5147. Correct reference to agent list in Chukwa bin scripts. (Ari
+ Rabkin via cdouglas)
+
+ HADOOP-5148. Fix logic disabling watchdog timer in Chukwa daemon scripts.
+ (Ari Rabkin via cdouglas)
+
+ HADOOP-5100. Append, rather than truncate, when creating log4j metrics in
+ Chukwa. (Jerome Boulon via cdouglas)
+
+ HADOOP-5204. Fix broken trunk compilation on Hudson by letting
+ task-controller be an independent target in build.xml.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-5212. Fix the path translation problem introduced by HADOOP-4868
+ running on cygwin. (Sharad Agarwal via omalley)
+
+ HADOOP-5226. Add license headers to html and jsp files. (szetszwo)
+
+ HADOOP-5172. Disable misbehaving Chukwa unit test until it can be fixed.
+ (Jerome Boulon via nigel)
+
+ HADOOP-4933. Fixes a ConcurrentModificationException problem that shows up
+ when the history viewer is accessed concurrently.
+ (Amar Kamat via ddas)
+
+ HADOOP-5253. Remove duplicate call to cn-docs target.
+ (Giri Kesavan via nigel)
+
+ HADOOP-5251. Fix classpath for contrib unit tests to include clover jar.
+ (nigel)
+
+ HADOOP-5206. Synchronize "unprotected*" methods of FSDirectory on the root.
+ (Jakob Homan via shv)
+
+ HADOOP-5292. Fix NPE in KFS::getBlockLocations. (Sriram Rao via lohit)
+
+ HADOOP-5219. Adds a new property io.seqfile.local.dir for use by SequenceFile,
+ which earlier used mapred.local.dir. (Sharad Agarwal via ddas)
+
+ HADOOP-5300. Fix ant javadoc-dev target and the typo in the class name
+ NameNodeActivtyMBean. (szetszwo)
+
+ HADOOP-5218. libhdfs unit test failed because it was unable to
+ start namenode/datanode. Fixed. (dhruba)
+
+ HADOOP-5273. Add license header to TestJobInProgress.java. (Jakob Homan
+ via szetszwo)
+
+ HADOOP-5229. Remove duplicate version variables in build files
+ (Stefan Groschupf via johan)
+
+ HADOOP-5383. Avoid building an unused string in NameNode's
+ verifyReplication(). (Raghu Angadi)
+
+ HADOOP-5347. Create a job output directory for the bbp examples. (szetszwo)
+
+ HADOOP-5341. Make hadoop-daemon scripts backwards compatible with the
+ changes in HADOOP-4868. (Sharad Agarwal via yhemanth)
+
+ HADOOP-5456. Fix javadoc links to ClientProtocol#restoreFailedStorage(..).
+ (Boris Shkolnik via szetszwo)
+
+ HADOOP-5458. Remove leftover Chukwa entries from build, etc. (cdouglas)
+
+ HADOOP-5386. Modify hdfsproxy unit test to start on a random port,
+ implement clover instrumentation. (Zhiyong Zhang via cdouglas)
+
+ HADOOP-5511. Add Apache License to EditLogBackupOutputStream. (shv)
+
+ HADOOP-5507. Fix JMXGet javadoc warnings. (Boris Shkolnik via szetszwo)
+
+ HADOOP-5191. Accessing HDFS with any ip or hostname should work as long
+ as it points to the interface NameNode is listening on. (Raghu Angadi)
+
+ HADOOP-5561. Add javadoc.maxmemory parameter to build, preventing OOM
+ exceptions from javadoc-dev. (Jakob Homan via cdouglas)
+
+ HADOOP-5149. Modify HistoryViewer to ignore unfamiliar files in the log
+ directory. (Hong Tang via cdouglas)
+
+ HADOOP-5477. Fix rare failure in TestCLI for hosts returning variations of
+ 'localhost'. (Jakob Homan via cdouglas)
+
+ HADOOP-5194. Disables setsid for tasks run on cygwin.
+ (Ravi Gummadi via ddas)
+
+ HADOOP-5322. Fix misleading/outdated comments in JobInProgress.
+ (Amareshwari Sriramadasu via cdouglas)
+
+ HADOOP-5198. Fixes a problem to do with the task PID file being absent and
+ the JvmManager trying to look for it. (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-5464. DFSClient did not treat write timeout of 0 properly.
+ (Raghu Angadi)
+
+ HADOOP-4045. Fix processing of IO errors in EditsLog.
+ (Boris Shkolnik via shv)
+
+ HADOOP-5462. Fixed a double free bug in the task-controller
+ executable. (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-5652. Fix a bug where in-memory segments are incorrectly retained in
+ memory. (cdouglas)
+
+ HADOOP-5533. Recovery duration shown on the jobtracker webpage is
+ inaccurate. (Amar Kamat via sharad)
+
+ HADOOP-5647. Fix TestJobHistory to not depend on /tmp. (Ravi Gummadi
+ via sharad)
+
+ HADOOP-5661. Fixes some findbugs warnings in o.a.h.mapred* packages and
+ supresses a bunch of them. (Jothi Padmanabhan via ddas)
+
+ HADOOP-5704. Fix compilation problems in TestFairScheduler and
+ TestCapacityScheduler. (Chris Douglas via szetszwo)
+
+ HADOOP-5650. Fix safemode messages in the Namenode log. (Suresh Srinivas
+ via szetszwo)
+
+ HADOOP-5488. Removes the pidfile management for the Task JVM from the
+ framework and instead passes the PID back and forth between the
+ TaskTracker and the Task processes. (Ravi Gummadi via ddas)
+
+ HADOOP-5658. Fix Eclipse templates. (Philip Zeyliger via shv)
+
+ HADOOP-5709. Remove redundant synchronization added in HADOOP-5661. (Jothi
+ Padmanabhan via cdouglas)
+
+ HADOOP-5715. Add conf/mapred-queue-acls.xml to the ignore lists.
+ (szetszwo)
+
+ HADOOP-5612. Some c++ scripts are not chmodded before ant execution.
+ (Todd Lipcon via tomwhite)
+
+ HADOOP-5611. Fix C++ libraries to build on Debian Lenny. (Todd Lipcon
+ via tomwhite)
+
+ HADOOP-5592. Fix typo in Streaming doc in reference to GzipCodec.
+ (Corinne Chandel via tomwhite)
+
+ HADOOP-5656. Counter for S3N Read Bytes does not work. (Ian Nowland
+ via tomwhite)
+
+ HADOOP-5406. Fix JNI binding for ZlibCompressor::setDictionary. (Lars
+ Francke via cdouglas)
+
+ HADOOP-3426. Fix/provide handling when DNS lookup fails on the loopback
+ address. Also cache the result of the lookup. (Steve Loughran via cdouglas)
+
+ HADOOP-5476. Close the underlying InputStream in SequenceFile::Reader when
+ the constructor throws an exception. (Michael Tamm via cdouglas)
+
+ HADOOP-5675. Do not launch a job if DistCp has no work to do. (Tsz Wo
+ (Nicholas), SZE via cdouglas)
+
+ HADOOP-5737. Fixes a problem in the way the JobTracker used to talk to
+ other daemons like the NameNode to get the job's files. Also adds APIs
+ in the JobTracker to get the FileSystem objects as per the JobTracker's
+ configuration. (Amar Kamat via ddas)
+
+ HADOOP-5648. Not able to generate gridmix.jar on the already compiled version of hadoop.
+ (gkesavan)
+
+ HADOOP-5808. Fix import never used javac warnings in hdfs. (szetszwo)
+
+ HADOOP-5203. TT's version build is too restrictive. (Rick Cox via sharad)
+
+ HADOOP-5818. Revert the renaming from FSNamesystem.checkSuperuserPrivilege
+ to checkAccess by HADOOP-5643. (Amar Kamat via szetszwo)
+
+ HADOOP-5820. Fix findbugs warnings for http related codes in hdfs.
+ (szetszwo)
+
+ HADOOP-5822. Fix javac warnings in several dfs tests related to unncessary
+ casts. (Jakob Homan via szetszwo)
+
+ HADOOP-5842. Fix a few javac warnings under packages fs and util.
+ (Hairong Kuang via szetszwo)
+
+ HADOOP-5845. Build successful despite test failure on test-core target.
+ (sharad)
+
+ HADOOP-5314. Prevent unnecessary saving of the file system image during
+ name-node startup. (Jakob Homan via shv)
+
+ HADOOP-5855. Fix javac warnings for DisallowedDatanodeException and
+ UnsupportedActionException. (szetszwo)
+
+ HADOOP-5582. Fixes a problem in Hadoop Vaidya to do with reading
+ counters from job history files. (Suhas Gogate via ddas)
+
+ HADOOP-5829. Fix javac warnings found in ReplicationTargetChooser,
+ FSImage, Checkpointer, SecondaryNameNode and a few other hdfs classes.
+ (Suresh Srinivas via szetszwo)
+
+ HADOOP-5835. Fix findbugs warnings found in Block, DataNode, NameNode and
+ a few other hdfs classes. (Suresh Srinivas via szetszwo)
+
+ HADOOP-5853. Undeprecate HttpServer.addInternalServlet method. (Suresh
+ Srinivas via szetszwo)
+
+ HADOOP-5801. Fixes the problem: If the hosts file is changed across restart
+ then it should be refreshed upon recovery so that the excluded hosts are
+ lost and the maps are re-executed. (Amar Kamat via ddas)
+
+ HADOOP-5841. Resolve findbugs warnings in DistributedFileSystem,
+ DatanodeInfo, BlocksMap, DataNodeDescriptor. (Jakob Homan via szetszwo)
+
+ HADOOP-5878. Fix import and Serializable javac warnings found in hdfs jsp.
+ (szetszwo)
+
+ HADOOP-5782. Revert a few formatting changes introduced in HADOOP-5015.
+ (Suresh Srinivas via rangadi)
+
+ HADOOP-5687. NameNode throws NPE if fs.default.name is the default value.
+ (Philip Zeyliger via shv)
+
+ HADOOP-5867. Fix javac warnings found in NNBench and NNBenchWithoutMR.
+ (Konstantin Boudnik via szetszwo)
+
+ HADOOP-5728. Fixed FSEditLog.printStatistics IndexOutOfBoundsException.
+ (Wang Xu via johan)
+
+ HADOOP-5847. Fixed failing Streaming unit tests (gkesavan)
+
+ HADOOP-5252. Streaming overrides -inputformat option (Klaas Bosteels
+ via sharad)
+
+ HADOOP-5710. Counter MAP_INPUT_BYTES missing from new mapreduce api.
+ (Amareshwari Sriramadasu via sharad)
+
+ HADOOP-5809. Fix job submission, broken by errant directory creation.
+ (Sreekanth Ramakrishnan and Jothi Padmanabhan via cdouglas)
+
+ HADOOP-5759. Fix for IllegalArgumentException when
+ CombineFileInputFormat is used as job InputFormat.
+ (Amareshwari Sriramadasu via dhruba)
+
+ HADOOP-5635. Change distributed cache to work with other distributed file
+ systems. (Andrew Hitchcock via tomwhite)
+
+ HADOOP-5856. Fix "unsafe multithreaded use of DateFormat" findbugs warning
+ in DataBlockScanner. (Kan Zhang via szetszwo)
+
+ HADOOP-4864. Fixes a problem to do with -libjars with multiple jars when
+ client and cluster reside on different OSs. (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-5623. Fixes a problem to do with status messages getting overwritten
+ in streaming jobs. (Rick Cox and Jothi Padmanabhan via ddas)
+
+ HADOOP-5895. Fixes computation of count of merged bytes for logging.
+ (Ravi Gummadi via ddas)
+
+ HADOOP-5805. problem using top level s3 buckets as input/output directories.
+ (Ian Nowland via tomwhite)
+
+ HADOOP-5940. trunk eclipse-plugin build fails while trying to copy
+ commons-cli jar from the lib dir (Giridharan Kesavan via gkesavan)
+
+ HADOOP-5864. Fix DMI and OBL findbugs in packages hdfs and metrics.
+ (hairong)
+
+ HADOOP-5935. Fix Hudson's release audit warnings link is broken.
+ (Giridharan Kesavan via gkesavan)
+
+ HADOOP-5947. Delete empty TestCombineFileInputFormat.java
+
+ HADOOP-5899. Move a log message in FSEditLog to the right place for
+ avoiding unnecessary log. (Suresh Srinivas via szetszwo)
+
+ HADOOP-5944. Add Apache license header to BlockManager.java. (Suresh
+ Srinivas via szetszwo)
+
+ HADOOP-5891. SecondaryNamenode is able to converse with the NameNode
+ even when the default value of dfs.http.address is not overridden.
+ (Todd Lipcon via dhruba)
+
+ HADOOP-5953. The isDirectory(..) and isFile(..) methods in KosmosFileSystem
+ should not be deprecated. (szetszwo)
+
+ HADOOP-5954. Fix javac warnings in TestFileCreation, TestSmallBlock,
+ TestFileStatus, TestDFSShellGenericOptions, TestSeekBug and
+ TestDFSStartupVersions. (szetszwo)
+
+ HADOOP-5956. Fix ivy dependency in hdfsproxy and capacity-scheduler.
+ (Giridharan Kesavan via szetszwo)
+
+ HADOOP-5836. Bug in S3N handling of directory markers using an object with
+ a trailing "/" causes jobs to fail. (Ian Nowland via tomwhite)
+
+ HADOOP-5861. s3n files are not getting split by default. (tomwhite)
+
+ HADOOP-5762. Fix a problem that DistCp does not copy empty directory.
+ (Rodrigo Schmidt via szetszwo)
+
+ HADOOP-5859. Fix "wait() or sleep() with locks held" findbugs warnings in
+ DFSClient. (Kan Zhang via szetszwo)
+
+ HADOOP-5457. Fix to continue to run builds even if contrib test fails
+ (Giridharan Kesavan via gkesavan)
+
+ HADOOP-5963. Remove an unnecessary exception catch in NNBench. (Boris
+ Shkolnik via szetszwo)
+
+ HADOOP-5989. Fix streaming test failure. (gkesavan)
+
+ HADOOP-5981. Fix a bug in HADOOP-2838 in parsing mapred.child.env.
+ (Amar Kamat via sharad)
+
+ HADOOP-5420. Fix LinuxTaskController to kill tasks using the process
+ groups they are launched with.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-6031. Remove @author tags from Java source files. (Ravi Phulari
+ via szetszwo)
+
+ HADOOP-5980. Fix LinuxTaskController so tasks get passed
+ LD_LIBRARY_PATH and other environment variables.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-4041. IsolationRunner does not work as documented.
+ (Philip Zeyliger via tomwhite)
+
+ HADOOP-6004. Fixes BlockLocation deserialization. (Jakob Homan via
+ szetszwo)
+
+ HADOOP-6079. Serialize proxySource as DatanodeInfo in DataTransferProtocol.
+ (szetszwo)
+
+Release 0.20.1 - Unreleased
+
+ INCOMPATIBLE CHANGES
+
+ HADOOP-5726. Remove pre-emption from capacity scheduler code base.
+ (Rahul Kumar Singh via yhemanth)
+
+ HADOOP-5881. Simplify memory monitoring and scheduling related
+ configuration. (Vinod Kumar Vavilapalli via yhemanth)
+
+ NEW FEATURES
+
+ IMPROVEMENTS
+
+ HADOOP-5711. Change Namenode file close log to info. (szetszwo)
+
+ HADOOP-5736. Update the capacity scheduler documentation for features
+ like memory based scheduling, job initialization and removal of pre-emption.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-5714. Add a metric for NameNode getFileInfo operation. (Jakob Homan
+ via szetszwo)
+
+ HADOOP-4372. Improves the way history filenames are obtained and manipulated.
+ (Amar Kamat via ddas)
+
+ OPTIMIZATIONS
+
+ BUG FIXES
+
+ HADOOP-5691. Makes org.apache.hadoop.mapreduce.Reducer concrete class
+ instead of abstract. (Amareshwari Sriramadasu via sharad)
+
+ HADOOP-5646. Fixes a problem in TestQueueCapacities.
+ (Vinod Kumar Vavilapalli via ddas)
+
+ HADOOP-5655. TestMRServerPorts fails on java.net.BindException. (Devaraj
+ Das via hairong)
+
+ HADOOP-5654. TestReplicationPolicy.<init> fails on java.net.BindException.
+ (hairong)
+
+ HADOOP-5688. Fix HftpFileSystem checksum path construction. (Tsz Wo
+ (Nicholas) Sze via cdouglas)
+
+ HADOOP-4674. Fix fs help messages for -test, -text, -tail, -stat
+ and -touchz options. (Ravi Phulari via szetszwo)
+
+ HADOOP-5718. Remove the check for the default queue in capacity scheduler.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-5719. Remove jobs that failed initialization from the waiting queue
+ in the capacity scheduler. (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-4744. Attaching another fix to the jetty port issue. The TaskTracker
+ kills itself if it ever discovers that the port to which jetty is actually
+ bound is invalid (-1). (ddas)
+
+ HADOOP-5349. Fixes a problem in LocalDirAllocator to check for the return
+ path value that is returned for the case where the file we want to write
+ is of an unknown size. (Vinod Kumar Vavilapalli via ddas)
+
+ HADOOP-5636. Prevents a job from going to RUNNING state after it has been
+ KILLED (this used to happen when the SetupTask would come back with a
+ success after the job has been killed). (Amar Kamat via ddas)
+
+ HADOOP-5641. Fix a NullPointerException in capacity scheduler's memory
+ based scheduling code when jobs get retired. (yhemanth)
+
+ HADOOP-5828. Use absolute path for mapred.local.dir of JobTracker in
+ MiniMRCluster. (yhemanth)
+
+ HADOOP-4981. Fix capacity scheduler to schedule speculative tasks
+ correctly in the presence of High RAM jobs.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-5210. Solves a problem in the progress report of the reduce task.
+ (Ravi Gummadi via ddas)
+
+ HADOOP-5850. Fixes a problem to do with not being able to jobs with
+ 0 maps/reduces. (Vinod K V via ddas)
+
+ HADOOP-4626. Correct the API links in hdfs forrest doc so that they
+ point to the same version of hadoop. (szetszwo)
+
+ HADOOP-5883. Fixed tasktracker memory monitoring to account for
+ momentary spurts in memory usage due to java's fork() model.
+ (yhemanth)
+
+ HADOOP-5539. Fixes a problem to do with not preserving intermediate
+ output compression for merged data.
+ (Jothi Padmanabhan and Billy Pearson via ddas)
+
+ HADOOP-5932. Fixes a problem in capacity scheduler in computing
+ available memory on a tasktracker.
+ (Vinod Kumar Vavilapalli via yhemanth)
+
+ HADOOP-5908. Fixes a problem to do with ArithmeticException in the
+ JobTracker when there are jobs with 0 maps. (Amar Kamat via ddas)
+
+ HADOOP-5924. Fixes a corner case problem to do with job recovery with
+ empty history files. Also, after a JT restart, sends KillTaskAction to
+ tasks that report back but the corresponding job hasn't been initialized
+ yet. (Amar Kamat via ddas)
+
+ HADOOP-5882. Fixes a reducer progress update problem for new mapreduce
+ api. (Amareshwari Sriramadasu via sharad)
+
+ HADOOP-5746. Fixes a corner case problem in Streaming, where if an exception
+ happens in MROutputThread after the last call to the map/reduce method, the
+ exception goes undetected. (Amar Kamat via ddas)
+
+ HADOOP-5884. Fixes accounting in capacity scheduler so that high RAM jobs
+ take more slots. (Vinod Kumar Vavilapalli via yhemanth)
+
+ HADOOP-5937. Correct a safemode message in FSNamesystem. (Ravi Phulari
+ via szetszwo)
+
+ HADOOP-5869. Fix bug in assignment of setup / cleanup task that was
+ causing TestQueueCapacities to fail.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-5921. Fixes a problem in the JobTracker where it sometimes never used
+ to come up due to a system file creation on JobTracker's system-dir failing.
+ This problem would sometimes show up only when the FS for the system-dir
+ (usually HDFS) is started at nearly the same time as the JobTracker.
+ (Amar Kamat via ddas)
+
+ HADOOP-5920. Fixes a testcase failure for TestJobHistory.
+ (Amar Kamat via ddas)
+
+Release 0.20.0 - 2009-04-15
+
+ INCOMPATIBLE CHANGES
+
+ HADOOP-4210. Fix findbugs warnings for equals implementations of mapred ID
+ classes. Removed public, static ID::read and ID::forName; made ID an
+ abstract class. (Suresh Srinivas via cdouglas)
+
+ HADOOP-4253. Fix various warnings generated by findbugs.
+ Following deprecated methods in RawLocalFileSystem are removed:
+ public String getName()
+ public void lock(Path p, boolean shared)
+ public void release(Path p)
+ (Suresh Srinivas via johan)
+
+ HADOOP-4618. Move http server from FSNamesystem into NameNode.
+ FSNamesystem.getNameNodeInfoPort() is removed.
+ FSNamesystem.getDFSNameNodeMachine() and FSNamesystem.getDFSNameNodePort()
+ replaced by FSNamesystem.getDFSNameNodeAddress().
+ NameNode(bindAddress, conf) is removed.
+ (shv)
+
+ HADOOP-4567. GetFileBlockLocations returns the NetworkTopology
+ information of the machines where the blocks reside. (dhruba)
+
+ HADOOP-4435. The JobTracker WebUI displays the amount of heap memory
+ in use. (dhruba)
+
+ HADOOP-4628. Move Hive into a standalone subproject. (omalley)
+
+ HADOOP-4188. Removes task's dependency on concrete filesystems.
+ (Sharad Agarwal via ddas)
+
+ HADOOP-1650. Upgrade to Jetty 6. (cdouglas)
+
+ HADOOP-3986. Remove static Configuration from JobClient. (Amareshwari
+ Sriramadasu via cdouglas)
+ JobClient::setCommandLineConfig is removed
+ JobClient::getCommandLineConfig is removed
+ JobShell, TestJobShell classes are removed
+
+ HADOOP-4422. S3 file systems should not create bucket.
+ (David Phillips via tomwhite)
+
+ HADOOP-4035. Support memory based scheduling in capacity scheduler.
+ (Vinod Kumar Vavilapalli via yhemanth)
+
+ HADOOP-3497. Fix bug in overly restrictive file globbing with a
+ PathFilter. (tomwhite)
+
+ HADOOP-4445. Replace running task counts with running task
+ percentage in capacity scheduler UI. (Sreekanth Ramakrishnan via
+ yhemanth)
+
+ HADOOP-4631. Splits the configuration into three parts - one for core,
+ one for mapred and the last one for HDFS. (Sharad Agarwal via cdouglas)
+
+ HADOOP-3344. Fix libhdfs build to use autoconf and build the same
+ architecture (32 vs 64 bit) of the JVM running Ant. The libraries for
+ pipes, utils, and libhdfs are now all in c++/<os_osarch_jvmdatamodel>/lib.
+ (Giridharan Kesavan via nigel)
+
+ HADOOP-4874. Remove LZO codec because of licensing issues. (omalley)
+
+ HADOOP-4970. The full path name of a file is preserved inside Trash.
+ (Prasad Chakka via dhruba)
+
+ HADOOP-4103. NameNode keeps a count of missing blocks. It warns on
+ WebUI if there are such blocks. '-report' and '-metaSave' have extra
+ info to track such blocks. (Raghu Angadi)
+
+ HADOOP-4783. Change permissions on history files on the jobtracker
+ to be only group readable instead of world readable.
+ (Amareshwari Sriramadasu via yhemanth)
+
+ NEW FEATURES
+
+ HADOOP-4575. Add a proxy service for relaying HsftpFileSystem requests.
+ Includes client authentication via user certificates and config-based
+ access control. (Kan Zhang via cdouglas)
+
+ HADOOP-4661. Add DistCh, a new tool for distributed ch{mod,own,grp}.
+ (szetszwo)
+
+ HADOOP-4709. Add several new features and bug fixes to Chukwa.
+ Added Hadoop Infrastructure Care Center (UI for visualize data collected
+ by Chukwa)
+ Added FileAdaptor for streaming small file in one chunk
+ Added compression to archive and demux output
+ Added unit tests and validation for agent, collector, and demux map
+ reduce job
+ Added database loader for loading demux output (sequence file) to jdbc
+ connected database
+ Added algorithm to distribute collector load more evenly
+ (Jerome Boulon, Eric Yang, Andy Konwinski, Ariel Rabkin via cdouglas)
+
+ HADOOP-4179. Add Vaidya tool to analyze map/reduce job logs for performanc
+ problems. (Suhas Gogate via omalley)
+
+ HADOOP-4029. Add NameNode storage information to the dfshealth page and
+ move DataNode information to a separated page. (Boris Shkolnik via
+ szetszwo)
+
+ HADOOP-4348. Add service-level authorization for Hadoop. (acmurthy)
+
+ HADOOP-4826. Introduce admin command saveNamespace. (shv)
+
+ HADOOP-3063 BloomMapFile - fail-fast version of MapFile for sparsely
+ populated key space (Andrzej Bialecki via stack)
+
+ HADOOP-1230. Add new map/reduce API and deprecate the old one. Generally,
+ the old code should work without problem. The new api is in
+ org.apache.hadoop.mapreduce and the old classes in org.apache.hadoop.mapred
+ are deprecated. Differences in the new API:
+ 1. All of the methods take Context objects that allow us to add new
+ methods without breaking compatability.
+ 2. Mapper and Reducer now have a "run" method that is called once and
+ contains the control loop for the task, which lets applications
+ replace it.
+ 3. Mapper and Reducer by default are Identity Mapper and Reducer.
+ 4. The FileOutputFormats use part-r-00000 for the output of reduce 0 and
+ part-m-00000 for the output of map 0.
+ 5. The reduce grouping comparator now uses the raw compare instead of
+ object compare.
+ 6. The number of maps in FileInputFormat is controlled by min and max
+ split size rather than min size and the desired number of maps.
+ (omalley)
+
+ HADOOP-3305. Use Ivy to manage dependencies. (Giridharan Kesavan
+ and Steve Loughran via cutting)
+
+ IMPROVEMENTS
+
+ HADOOP-4749. Added a new counter REDUCE_INPUT_BYTES. (Yongqiang He via
+ zshao)
+
+ HADOOP-4234. Fix KFS "glue" layer to allow applications to interface
+ with multiple KFS metaservers. (Sriram Rao via lohit)
+
+ HADOOP-4245. Update to latest version of KFS "glue" library jar.
+ (Sriram Rao via lohit)
+
+ HADOOP-4244. Change test-patch.sh to check Eclipse classpath no matter
+ it is run by Hudson or not. (szetszwo)
+
+ HADOOP-3180. Add name of missing class to WritableName.getClass
+ IOException. (Pete Wyckoff via omalley)
+
+ HADOOP-4178. Make the capacity scheduler's default values configurable.
+ (Sreekanth Ramakrishnan via omalley)
+
+ HADOOP-4262. Generate better error message when client exception has null
+ message. (stevel via omalley)
+
+ HADOOP-4226. Refactor and document LineReader to make it more readily
+ understandable. (Yuri Pradkin via cdouglas)
+
+ HADOOP-4238. When listing jobs, if scheduling information isn't available
+ print NA instead of empty output. (Sreekanth Ramakrishnan via johan)
+
+ HADOOP-4284. Support filters that apply to all requests, or global filters,
+ to HttpServer. (Kan Zhang via cdouglas)
+
+ HADOOP-4276. Improve the hashing functions and deserialization of the
+ mapred ID classes. (omalley)
+
+ HADOOP-4485. Add a compile-native ant task, as a shorthand. (enis)
+
+ HADOOP-4454. Allow # comments in slaves file. (Rama Ramasamy via omalley)
+
+ HADOOP-3461. Remove hdfs.StringBytesWritable. (szetszwo)
+
+ HADOOP-4437. Use Halton sequence instead of java.util.Random in
+ PiEstimator. (szetszwo)
+
+ HADOOP-4572. Change INode and its sub-classes to package private.
+ (szetszwo)
+
+ HADOOP-4187. Does a runtime lookup for JobConf/JobConfigurable, and if
+ found, invokes the appropriate configure method. (Sharad Agarwal via ddas)
+
+ HADOOP-4453. Improve ssl configuration and handling in HsftpFileSystem,
+ particularly when used with DistCp. (Kan Zhang via cdouglas)
+
+ HADOOP-4583. Several code optimizations in HDFS. (Suresh Srinivas via
+ szetszwo)
+
+ HADOOP-3923. Remove org.apache.hadoop.mapred.StatusHttpServer. (szetszwo)
+
+ HADOOP-4622. Explicitly specify interpretor for non-native
+ pipes binaries. (Fredrik Hedberg via johan)
+
+ HADOOP-4505. Add a unit test to test faulty setup task and cleanup
+ task killing the job. (Amareshwari Sriramadasu via johan)
+
+ HADOOP-4608. Don't print a stack trace when the example driver gets an
+ unknown program to run. (Edward Yoon via omalley)
+
+ HADOOP-4645. Package HdfsProxy contrib project without the extra level
+ of directories. (Kan Zhang via omalley)
+
+ HADOOP-4126. Allow access to HDFS web UI on EC2 (tomwhite via omalley)
+
+ HADOOP-4612. Removes RunJar's dependency on JobClient.
+ (Sharad Agarwal via ddas)
+
+ HADOOP-4185. Adds setVerifyChecksum() method to FileSystem.
+ (Sharad Agarwal via ddas)
+
+ HADOOP-4523. Prevent too many tasks scheduled on a node from bringing
+ it down by monitoring for cumulative memory usage across tasks.
+ (Vinod Kumar Vavilapalli via yhemanth)
+
+ HADOOP-4640. Adds an input format that can split lzo compressed
+ text files. (johan)
+
+ HADOOP-4666. Launch reduces only after a few maps have run in the
+ Fair Scheduler. (Matei Zaharia via johan)
+
+ HADOOP-4339. Remove redundant calls from FileSystem/FsShell when
+ generating/processing ContentSummary. (David Phillips via cdouglas)
+
+ HADOOP-2774. Add counters tracking records spilled to disk in MapTask and
+ ReduceTask. (Ravi Gummadi via cdouglas)
+
+ HADOOP-4513. Initialize jobs asynchronously in the capacity scheduler.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-4649. Improve abstraction for spill indices. (cdouglas)
+
+ HADOOP-3770. Add gridmix2, an iteration on the gridmix benchmark. (Runping
+ Qi via cdouglas)
+
+ HADOOP-4708. Add support for dfsadmin commands in TestCLI. (Boris Shkolnik
+ via cdouglas)
+
+ HADOOP-4758. Add a splitter for metrics contexts to support more than one
+ type of collector. (cdouglas)
+
+ HADOOP-4722. Add tests for dfsadmin quota error messages. (Boris Shkolnik
+ via cdouglas)
+
+ HADOOP-4690. fuse-dfs - create source file/function + utils + config +
+ main source files. (pete wyckoff via mahadev)
+
+ HADOOP-3750. Fix and enforce module dependencies. (Sharad Agarwal via
+ tomwhite)
+
+ HADOOP-4747. Speed up FsShell::ls by removing redundant calls to the
+ filesystem. (David Phillips via cdouglas)
+
+ HADOOP-4305. Improves the blacklisting strategy, whereby, tasktrackers
+ that are blacklisted are not given tasks to run from other jobs, subject
+ to the following conditions (all must be met):
+ 1) The TaskTracker has been blacklisted by at least 4 jobs (configurable)
+ 2) The TaskTracker has been blacklisted 50% more number of times than
+ the average (configurable)
+ 3) The cluster has less than 50% trackers blacklisted
+ Once in 24 hours, a TaskTracker blacklisted for all jobs is given a chance.
+ Restarting the TaskTracker moves it out of the blacklist.
+ (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-4688. Modify the MiniMRDFSSort unit test to spill multiple times,
+ exercising the map-side merge code. (cdouglas)
+
+ HADOOP-4737. Adds the KILLED notification when jobs get killed.
+ (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-4728. Add a test exercising different namenode configurations.
+ (Boris Shkolnik via cdouglas)
+
+ HADOOP-4807. Adds JobClient commands to get the active/blacklisted tracker
+ names. Also adds commands to display running/completed task attempt IDs.
+ (ddas)
+
+ HADOOP-4699. Remove checksum validation from map output servlet. (cdouglas)
+
+ HADOOP-4838. Added a registry to automate metrics and mbeans management.
+ (Sanjay Radia via acmurthy)
+
+ HADOOP-3136. Fixed the default scheduler to assign multiple tasks to each
+ tasktracker per heartbeat, when feasible. To ensure locality isn't hurt
+ too badly, the scheudler will not assign more than one off-switch task per
+ heartbeat. The heartbeat interval is also halved since the task-tracker is
+ fixed to no longer send out heartbeats on each task completion. A
+ slow-start for scheduling reduces is introduced to ensure that reduces
+ aren't started till sufficient number of maps are done, else reduces of
+ jobs whose maps aren't scheduled might swamp the cluster.
+ Configuration changes to mapred-default.xml:
+ add mapred.reduce.slowstart.completed.maps
+ (acmurthy)
+
+ HADOOP-4545. Add example and test case of secondary sort for the reduce.
+ (omalley)
+
+ HADOOP-4753. Refactor gridmix2 to reduce code duplication. (cdouglas)
+
+ HADOOP-4909. Fix Javadoc and make some of the API more consistent in their
+ use of the JobContext instead of Configuration. (omalley)
+
+ HADOOP-4920. Stop storing Forrest output in Subversion. (cutting)
+
+ HADOOP-4948. Add parameters java5.home and forrest.home to the ant commands
+ in test-patch.sh. (Giridharan Kesavan via szetszwo)
+
+ HADOOP-4830. Add end-to-end test cases for testing queue capacities.
+ (Vinod Kumar Vavilapalli via yhemanth)
+
+ HADOOP-4980. Improve code layout of capacity scheduler to make it
+ easier to fix some blocker bugs. (Vivek Ratan via yhemanth)
+
+ HADOOP-4916. Make user/location of Chukwa installation configurable by an
+ external properties file. (Eric Yang via cdouglas)
+
+ HADOOP-4950. Make the CompressorStream, DecompressorStream,
+ BlockCompressorStream, and BlockDecompressorStream public to facilitate
+ non-Hadoop codecs. (omalley)
+
+ HADOOP-4843. Collect job history and configuration in Chukwa. (Eric Yang
+ via cdouglas)
+
+ HADOOP-5030. Build Chukwa RPM to install into configured directory. (Eric
+ Yang via cdouglas)
+
+ HADOOP-4828. Updates documents to do with configuration (HADOOP-4631).
+ (Sharad Agarwal via ddas)
+
+ HADOOP-4939. Adds a test that would inject random failures for tasks in
+ large jobs and would also inject TaskTracker failures. (ddas)
+
+ HADOOP-4944. A configuration file can include other configuration
+ files. (Rama Ramasamy via dhruba)
+
+ HADOOP-4804. Provide Forrest documentation for the Fair Scheduler.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-5248. A testcase that checks for the existence of job directory
+ after the job completes. Fails if it exists. (ddas)
+
+ HADOOP-4664. Introduces multiple job initialization threads, where the
+ number of threads are configurable via mapred.jobinit.threads.
+ (Matei Zaharia and Jothi Padmanabhan via ddas)
+
+ HADOOP-4191. Adds a testcase for JobHistory. (Ravi Gummadi via ddas)
+
+ HADOOP-5466. Change documenation CSS style for headers and code. (Corinne
+ Chandel via szetszwo)
+
+ HADOOP-5275. Add ivy directory and files to built tar.
+ (Giridharan Kesavan via nigel)
+
+ HADOOP-5468. Add sub-menus to forrest documentation and make some minor
+ edits. (Corinne Chandel via szetszwo)
+
+ HADOOP-5437. Fix TestMiniMRDFSSort to properly test jvm-reuse. (omalley)
+
+ HADOOP-5521. Removes dependency of TestJobInProgress on RESTART_COUNT
+ JobHistory tag. (Ravi Gummadi via ddas)
+
+ OPTIMIZATIONS
+
+ HADOOP-3293. Fixes FileInputFormat to do provide locations for splits
+ based on the rack/host that has the most number of bytes.
+ (Jothi Padmanabhan via ddas)
+
+ HADOOP-4683. Fixes Reduce shuffle scheduler to invoke
+ getMapCompletionEvents in a separate thread. (Jothi Padmanabhan
+ via ddas)
+
+ BUG FIXES
+
+ HADOOP-4204. Fix findbugs warnings related to unused variables, naive
+ Number subclass instantiation, Map iteration, and badly scoped inner
+ classes. (Suresh Srinivas via cdouglas)
+
+ HADOOP-4207. Update derby jar file to release 10.4.2 release.
+ (Prasad Chakka via dhruba)
+
+ HADOOP-4325. SocketInputStream.read() should return -1 in case EOF.
+ (Raghu Angadi)
+
+ HADOOP-4408. FsAction functions need not create new objects. (cdouglas)
+
+ HADOOP-4440. TestJobInProgressListener tests for jobs killed in queued
+ state (Amar Kamat via ddas)
+
+ HADOOP-4346. Implement blocking connect so that Hadoop is not affected
+ by selector problem with JDK default implementation. (Raghu Angadi)
+
+ HADOOP-4388. If there are invalid blocks in the transfer list, Datanode
+ should handle them and keep transferring the remaining blocks. (Suresh
+ Srinivas via szetszwo)
+
+ HADOOP-4587. Fix a typo in Mapper javadoc. (Koji Noguchi via szetszwo)
+
+ HADOOP-4530. In fsck, HttpServletResponse sendError fails with
+ IllegalStateException. (hairong)
+
+ HADOOP-4377. Fix a race condition in directory creation in
+ NativeS3FileSystem. (David Phillips via cdouglas)
+
+ HADOOP-4621. Fix javadoc warnings caused by duplicate jars. (Kan Zhang via
+ cdouglas)
+
+ HADOOP-4566. Deploy new hive code to support more types.
+ (Zheng Shao via dhruba)
+
+ HADOOP-4571. Add chukwa conf files to svn:ignore list. (Eric Yang via
+ szetszwo)
+
+ HADOOP-4589. Correct PiEstimator output messages and improve the code
+ readability. (szetszwo)
+
+ HADOOP-4650. Correct a mismatch between the default value of
+ local.cache.size in the config and the source. (Jeff Hammerbacher via
+ cdouglas)
+
+ HADOOP-4606. Fix cygpath error if the log directory does not exist.
+ (szetszwo via omalley)
+
+ HADOOP-4141. Fix bug in ScriptBasedMapping causing potential infinite
+ loop on misconfigured hadoop-site. (Aaron Kimball via tomwhite)
+
+ HADOOP-4691. Correct a link in the javadoc of IndexedSortable. (szetszwo)
+
+ HADOOP-4598. '-setrep' command skips under-replicated blocks. (hairong)
+
+ HADOOP-4429. Set defaults for user, group in UnixUserGroupInformation so
+ login fails more predictably when misconfigured. (Alex Loddengaard via
+ cdouglas)
+
+ HADOOP-4676. Fix broken URL in blacklisted tasktrackers page. (Amareshwari
+ Sriramadasu via cdouglas)
+
+ HADOOP-3422 Ganglia counter metrics are all reported with the metric
+ name "value", so the counter values can not be seen. (Jason Attributor
+ and Brian Bockelman via stack)
+
+ HADOOP-4704. Fix javadoc typos "the the". (szetszwo)
+
+ HADOOP-4677. Fix semantics of FileSystem::getBlockLocations to return
+ meaningful values. (Hong Tang via cdouglas)
+
+ HADOOP-4669. Use correct operator when evaluating whether access time is
+ enabled (Dhruba Borthakur via cdouglas)
+
+ HADOOP-4732. Pass connection and read timeouts in the correct order when
+ setting up fetch in reduce. (Amareshwari Sriramadasu via cdouglas)
+
+ HADOOP-4558. Fix capacity reclamation in capacity scheduler.
+ (Amar Kamat via yhemanth)
+
+ HADOOP-4770. Fix rungridmix_2 script to work with RunJar. (cdouglas)
+
+ HADOOP-4738. When using git, the saveVersion script will use only the
+ commit hash for the version and not the message, which requires escaping.
+ (cdouglas)
+
+ HADOOP-4576. Show pending job count instead of task count in the UI per
+ queue in capacity scheduler. (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-4623. Maintain running tasks even if speculative execution is off.
+ (Amar Kamat via yhemanth)
+
+ HADOOP-4786. Fix broken compilation error in
+ TestTrackerBlacklistAcrossJobs. (yhemanth)
+
+ HADOOP-4785. Fixes theJobTracker heartbeat to not make two calls to
+ System.currentTimeMillis(). (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-4792. Add generated Chukwa configuration files to version control
+ ignore lists. (cdouglas)
+
+ HADOOP-4796. Fix Chukwa test configuration, remove unused components. (Eric
+ Yang via cdouglas)
+
+ HADOOP-4708. Add binaries missed in the initial checkin for Chukwa. (Eric
+ Yang via cdouglas)
+
+ HADOOP-4805. Remove black list collector from Chukwa Agent HTTP Sender.
+ (Eric Yang via cdouglas)
+
+ HADOOP-4837. Move HADOOP_CONF_DIR configuration to chukwa-env.sh (Jerome
+ Boulon via cdouglas)
+
+ HADOOP-4825. Use ps instead of jps for querying process status in Chukwa.
+ (Eric Yang via cdouglas)
+
+ HADOOP-4844. Fixed javadoc for
+ org.apache.hadoop.fs.permission.AccessControlException to document that
+ it's deprecated in favour of
+ org.apache.hadoop.security.AccessControlException. (acmurthy)
+
+ HADOOP-4706. Close the underlying output stream in
+ IFileOutputStream::close. (Jothi Padmanabhan via cdouglas)
+
+ HADOOP-4855. Fixed command-specific help messages for refreshServiceAcl in
+ DFSAdmin and MRAdmin. (acmurthy)
+
+ HADOOP-4820. Remove unused method FSNamesystem::deleteInSafeMode. (Suresh
+ Srinivas via cdouglas)
+
+ HADOOP-4698. Lower io.sort.mb to 10 in the tests and raise the junit memory
+ limit to 512m from 256m. (Nigel Daley via cdouglas)
+
+ HADOOP-4860. Split TestFileTailingAdapters into three separate tests to
+ avoid contention. (Eric Yang via cdouglas)
+
+ HADOOP-3921. Fixed clover (code coverage) target to work with JDK 6.
+ (tomwhite via nigel)
+
+ HADOOP-4845. Modify the reduce input byte counter to record only the
+ compressed size and add a human-readable label. (Yongqiang He via cdouglas)
+
+ HADOOP-4458. Add a test creating symlinks in the working directory.
+ (Amareshwari Sriramadasu via cdouglas)
+
+ HADOOP-4879. Fix org.apache.hadoop.mapred.Counters to correctly define
+ Object.equals rather than depend on contentEquals api. (omalley via
+ acmurthy)
+
+ HADOOP-4791. Fix rpm build process for Chukwa. (Eric Yang via cdouglas)
+
+ HADOOP-4771. Correct initialization of the file count for directories
+ with quotas. (Ruyue Ma via shv)
+
+ HADOOP-4878. Fix eclipse plugin classpath file to point to ivy's resolved
+ lib directory and added the same to test-patch.sh. (Giridharan Kesavan via
+ acmurthy)
+
+ HADOOP-4774. Fix default values of some capacity scheduler configuration
+ items which would otherwise not work on a fresh checkout.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-4876. Fix capacity scheduler reclamation by updating count of
+ pending tasks correctly. (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-4849. Documentation for Service Level Authorization implemented in
+ HADOOP-4348. (acmurthy)
+
+ HADOOP-4827. Replace Consolidator with Aggregator macros in Chukwa (Eric
+ Yang via cdouglas)
+
+ HADOOP-4894. Correctly parse ps output in Chukwa jettyCollector.sh. (Ari
+ Rabkin via cdouglas)
+
+ HADOOP-4892. Close fds out of Chukwa ExecPlugin. (Ari Rabkin via cdouglas)
+
+ HADOOP-4889. Fix permissions in RPM packaging. (Eric Yang via cdouglas)
+
+ HADOOP-4869. Fixes the TT-JT heartbeat to have an explicit flag for
+ restart apart from the initialContact flag that there was earlier.
+ (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-4716. Fixes ReduceTask.java to clear out the mapping between
+ hosts and MapOutputLocation upon a JT restart (Amar Kamat via ddas)
+
+ HADOOP-4880. Removes an unnecessary testcase from TestJobTrackerRestart.
+ (Amar Kamat via ddas)
+
+ HADOOP-4924. Fixes a race condition in TaskTracker re-init. (ddas)
+
+ HADOOP-4854. Read reclaim capacity interval from capacity scheduler
+ configuration. (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-4896. HDFS Fsck does not load HDFS configuration. (Raghu Angadi)
+
+ HADOOP-4956. Creates TaskStatus for failed tasks with an empty Counters
+ object instead of null. (ddas)
+
+ HADOOP-4979. Fix capacity scheduler to block cluster for failed high
+ RAM requirements across task types. (Vivek Ratan via yhemanth)
+
+ HADOOP-4949. Fix native compilation. (Chris Douglas via acmurthy)
+
+ HADOOP-4787. Fixes the testcase TestTrackerBlacklistAcrossJobs which was
+ earlier failing randomly. (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-4914. Add description fields to Chukwa init.d scripts (Eric Yang via
+ cdouglas)
+
+ HADOOP-4884. Make tool tip date format match standard HICC format. (Eric
+ Yang via cdouglas)
+
+ HADOOP-4925. Make Chukwa sender properties configurable. (Ari Rabkin via
+ cdouglas)
+
+ HADOOP-4947. Make Chukwa command parsing more forgiving of whitespace. (Ari
+ Rabkin via cdouglas)
+
+ HADOOP-5026. Make chukwa/bin scripts executable in repository. (Andy
+ Konwinski via cdouglas)
+
+ HADOOP-4977. Fix a deadlock between the reclaimCapacity and assignTasks
+ in capacity scheduler. (Vivek Ratan via yhemanth)
+
+ HADOOP-4988. Fix reclaim capacity to work even when there are queues with
+ no capacity. (Vivek Ratan via yhemanth)
+
+ HADOOP-5065. Remove generic parameters from argument to
+ setIn/OutputFormatClass so that it works with SequenceIn/OutputFormat.
+ (cdouglas via omalley)
+
+ HADOOP-4818. Pass user config to instrumentation API. (Eric Yang via
+ cdouglas)
+
+ HADOOP-4993. Fix Chukwa agent configuration and startup to make it both
+ more modular and testable. (Ari Rabkin via cdouglas)
+
+ HADOOP-5048. Fix capacity scheduler to correctly cleanup jobs that are
+ killed after initialization, but before running.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ HADOOP-4671. Mark loop control variables shared between threads as
+ volatile. (cdouglas)
+
+ HADOOP-5079. HashFunction inadvertently destroys some randomness
+ (Jonathan Ellis via stack)
+
+ HADOOP-4999. A failure to write to FsEditsLog results in
+ IndexOutOfBounds exception. (Boris Shkolnik via rangadi)
+
+ HADOOP-5139. Catch IllegalArgumentException during metrics registration
+ in RPC. (Hairong Kuang via szetszwo)
+
+ HADOOP-5085. Copying a file to local with Crc throws an exception.
+ (hairong)
+
+ HADOOP-5211. Fix check for job completion in TestSetupAndCleanupFailure.
+ (enis)
+
+ HADOOP-5254. The Configuration class should be able to work with XML
+ parsers that do not support xmlinclude. (Steve Loughran via dhruba)
+
+ HADOOP-4692. Namenode in infinite loop for replicating/deleting corrupt
+ blocks. (hairong)
+
+ HADOOP-5255. Fix use of Math.abs to avoid overflow. (Jonathan Ellis via
+ cdouglas)
+
+ HADOOP-5269. Fixes a problem to do with tasktracker holding on to
+ FAILED_UNCLEAN or KILLED_UNCLEAN tasks forever. (Amareshwari Sriramadasu
+ via ddas)
+
+ HADOOP-5214. Fixes a ConcurrentModificationException while the Fairshare
+ Scheduler accesses the tasktrackers stored by the JobTracker.
+ (Rahul Kumar Singh via yhemanth)
+
+ HADOOP-5233. Addresses the three issues - Race condition in updating
+ status, NPE in TaskTracker task localization when the conf file is missing
+ (HADOOP-5234) and NPE in handling KillTaskAction of a cleanup task
+ (HADOOP-5235). (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-5247. Introduces a broadcast of KillJobAction to all trackers when
+ a job finishes. This fixes a bunch of problems to do with NPE when a
+ completed job is not in memory and a tasktracker comes to the jobtracker
+ with a status report of a task belonging to that job. (Amar Kamat via ddas)
+
+ HADOOP-5282. Fixed job history logs for task attempts that are
+ failed by the JobTracker, say due to lost task trackers. (Amar
+ Kamat via yhemanth)
+
+ HADOOP-5241. Fixes a bug in disk-space resource estimation. Makes
+ the estimation formula linear where blowUp =
+ Total-Output/Total-Input. (Sharad Agarwal via ddas)
+
+ HADOOP-5142. Fix MapWritable#putAll to store key/value classes.
+ (Do??acan G??ney via enis)
+
+ HADOOP-4744. Workaround for jetty6 returning -1 when getLocalPort
+ is invoked on the connector. The workaround patch retries a few
+ times before failing. (Jothi Padmanabhan via yhemanth)
+
+ HADOOP-5280. Adds a check to prevent a task state transition from
+ FAILED to any of UNASSIGNED, RUNNING, COMMIT_PENDING or
+ SUCCEEDED. (ddas)
+
+ HADOOP-5272. Fixes a problem to do with detecting whether an
+ attempt is the first attempt of a Task. This affects JobTracker
+ restart. (Amar Kamat via ddas)
+
+ HADOOP-5306. Fixes a problem to do with logging/parsing the http port of a
+ lost tracker. Affects JobTracker restart. (Amar Kamat via ddas)
+
+ HADOOP-5111. Fix Job::set* methods to work with generics. (cdouglas)
+
+ HADOOP-5274. Fix gridmix2 dependency on wordcount example. (cdouglas)
+
+ HADOOP-5145. Balancer sometimes runs out of memory after running
+ days or weeks. (hairong)
+
+ HADOOP-5338. Fix jobtracker restart to clear task completion
+ events cached by tasktrackers forcing them to fetch all events
+ afresh, thus avoiding missed task completion events on the
+ tasktrackers. (Amar Kamat via yhemanth)
+
+ HADOOP-4695. Change TestGlobalFilter so that it allows a web page to be
+ filtered more than once for a single access. (Kan Zhang via szetszwo)
+
+ HADOOP-5298. Change TestServletFilter so that it allows a web page to be
+ filtered more than once for a single access. (szetszwo)
+
+ HADOOP-5432. Disable ssl during unit tests in hdfsproxy, as it is unused
+ and causes failures. (cdouglas)
+
+ HADOOP-5416. Correct the shell command "fs -test" forrest doc description.
+ (Ravi Phulari via szetszwo)
+
+ HADOOP-5327. Fixed job tracker to remove files from system directory on
+ ACL check failures and also check ACLs on restart.
+ (Amar Kamat via yhemanth)
+
+ HADOOP-5395. Change the exception message when a job is submitted to an
+ invalid queue. (Rahul Kumar Singh via yhemanth)
+
+ HADOOP-5276. Fixes a problem to do with updating the start time of
+ a task when the tracker that ran the task is lost. (Amar Kamat via
+ ddas)
+
+ HADOOP-5278. Fixes a problem to do with logging the finish time of
+ a task during recovery (after a JobTracker restart). (Amar Kamat
+ via ddas)
+
+ HADOOP-5490. Fixes a synchronization problem in the
+ EagerTaskInitializationListener class. (Jothi Padmanabhan via
+ ddas)
+
+ HADOOP-5493. The shuffle copier threads return the codecs back to
+ the pool when the shuffle completes. (Jothi Padmanabhan via ddas)
+
+ HADOOP-5414. Fixes IO exception while executing hadoop fs -touchz
+ fileName by making sure that lease renewal thread exits before dfs
+ client exits. (hairong)
+
+ HADOOP-5103. FileInputFormat now reuses the clusterMap network
+ topology object and that brings down the log messages in the
+ JobClient to do with NetworkTopology.add significantly. (Jothi
+ Padmanabhan via ddas)
+
+ HADOOP-5483. Fixes a problem in the Directory Cleanup Thread due to which
+ TestMiniMRWithDFS sometimes used to fail. (ddas)
+
+ HADOOP-5281. Prevent sharing incompatible ZlibCompressor instances between
+ GzipCodec and DefaultCodec. (cdouglas)
+
+ HADOOP-5463. Balancer throws "Not a host:port pair" unless port is
+ specified in fs.default.name. (Stuart White via hairong)
+
+ HADOOP-5514. Fix JobTracker metrics and add metrics for wating, failed
+ tasks. (cdouglas)
+
+ HADOOP-5516. Fix NullPointerException in TaskMemoryManagerThread
+ that comes when monitored processes disappear when the thread is
+ running. (Vinod Kumar Vavilapalli via yhemanth)
+
+ HADOOP-5382. Support combiners in the new context object API. (omalley)
+
+ HADOOP-5471. Fixes a problem to do with updating the log.index file in the
+ case where a cleanup task is run. (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-5534. Fixed a deadlock in Fair scheduler's servlet.
+ (Rahul Kumar Singh via yhemanth)
+
+ HADOOP-5328. Fixes a problem in the renaming of job history files during
+ job recovery. (Amar Kamat via ddas)
+
+ HADOOP-5417. Don't ignore InterruptedExceptions that happen when calling
+ into rpc. (omalley)
+
+ HADOOP-5320. Add a close() in TestMapReduceLocal. (Jothi Padmanabhan
+ via szetszwo)
+
+ HADOOP-5520. Fix a typo in disk quota help message. (Ravi Phulari
+ via szetszwo)
+
+ HADOOP-5519. Remove claims from mapred-default.xml that prime numbers
+ of tasks are helpful. (Owen O'Malley via szetszwo)
+
+ HADOOP-5484. TestRecoveryManager fails wtih FileAlreadyExistsException.
+ (Amar Kamat via hairong)
+
+ HADOOP-5564. Limit the JVM heap size in the java command for initializing
+ JAVA_PLATFORM. (Suresh Srinivas via szetszwo)
+
+ HADOOP-5565. Add API for failing/finalized jobs to the JT metrics
+ instrumentation. (Jerome Boulon via cdouglas)
+
+ HADOOP-5390. Remove duplicate jars from tarball, src from binary tarball
+ added by hdfsproxy. (Zhiyong Zhang via cdouglas)
+
+ HADOOP-5066. Building binary tarball should not build docs/javadocs, copy
+ src, or run jdiff. (Giridharan Kesavan via cdouglas)
+
+ HADOOP-5459. Fix undetected CRC errors where intermediate output is closed
+ before it has been completely consumed. (cdouglas)
+
+ HADOOP-5571. Remove widening primitive conversion in TupleWritable mask
+ manipulation. (Jingkei Ly via cdouglas)
+
+ HADOOP-5588. Remove an unnecessary call to listStatus(..) in
+ FileSystem.globStatusInternal(..). (Hairong Kuang via szetszwo)
+
+ HADOOP-5473. Solves a race condition in killing a task - the state is KILLED
+ if there is a user request pending to kill the task and the TT reported
+ the state as SUCCESS. (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-5576. Fix LocalRunner to work with the new context object API in
+ mapreduce. (Tom White via omalley)
+
+ HADOOP-4374. Installs a shutdown hook in the Task JVM so that log.index is
+ updated before the JVM exits. Also makes the update to log.index atomic.
+ (Ravi Gummadi via ddas)
+
+ HADOOP-5577. Add a verbose flag to mapreduce.Job.waitForCompletion to get
+ the running job's information printed to the user's stdout as it runs.
+ (omalley)
+
+ HADOOP-5607. Fix NPE in TestCapacityScheduler. (cdouglas)
+
+ HADOOP-5605. All the replicas incorrectly got marked as corrupt. (hairong)
+
+ HADOOP-5337. JobTracker, upon restart, now waits for the TaskTrackers to
+ join back before scheduling new tasks. This fixes race conditions associated
+ with greedy scheduling as was the case earlier. (Amar Kamat via ddas)
+
+ HADOOP-5227. Fix distcp so -update and -delete can be meaningfully
+ combined. (Tsz Wo (Nicholas), SZE via cdouglas)
+
+ HADOOP-5305. Increase number of files and print debug messages in
+ TestCopyFiles. (szetszwo)
+
+ HADOOP-5548. Add synchronization for JobTracker methods in RecoveryManager.
+ (Amareshwari Sriramadasu via sharad)
+
+ HADOOP-3810. NameNode seems unstable on a cluster with little space left.
+ (hairong)
+
+ HADOOP-5068. Fix NPE in TestCapacityScheduler. (Vinod Kumar Vavilapalli
+ via szetszwo)
+
+ HADOOP-5585. Clear FileSystem statistics between tasks when jvm-reuse
+ is enabled. (omalley)
+
+ HADOOP-5394. JobTracker might schedule 2 attempts of the same task
+ with the same attempt id across restarts. (Amar Kamat via sharad)
+
+ HADOOP-5645. After HADOOP-4920 we need a place to checkin
+ releasenotes.html. (nigel)
+
+Release 0.19.2 - Unreleased
+
+ BUG FIXES
+
+ HADOOP-5154. Fixes a deadlock in the fairshare scheduler.
+ (Matei Zaharia via yhemanth)
+
+ HADOOP-5146. Fixes a race condition that causes LocalDirAllocator to miss
+ files. (Devaraj Das via yhemanth)
+
+ HADOOP-4638. Fixes job recovery to not crash the job tracker for problems
+ with a single job file. (Amar Kamat via yhemanth)
+
+ HADOOP-5384. Fix a problem that DataNodeCluster creates blocks with
+ generationStamp == 1. (szetszwo)
+
+ HADOOP-5376. Fixes the code handling lost tasktrackers to set the task state
+ to KILLED_UNCLEAN only for relevant type of tasks.
+ (Amareshwari Sriramadasu via yhemanth)
+
+ HADOOP-5285. Fixes the issues - (1) obtainTaskCleanupTask checks whether job is
+ inited before trying to lock the JobInProgress (2) Moves the CleanupQueue class
+ outside the TaskTracker and makes it a generic class that is used by the
+ JobTracker also for deleting the paths on the job's output fs. (3) Moves the
+ references to completedJobStore outside the block where the JobTracker is locked.
+ (ddas)
+
+ HADOOP-5392. Fixes a problem to do with JT crashing during recovery when
+ the job files are garbled. (Amar Kamat via ddas)
+
+ HADOOP-5332. Appending to files is not allowed (by default) unless
+ dfs.support.append is set to true. (dhruba)
+
+ HADOOP-5333. libhdfs supports appending to files. (dhruba)
+
+ HADOOP-3998. Fix dfsclient exception when JVM is shutdown. (dhruba)
+
+ HADOOP-5440. Fixes a problem to do with removing a taskId from the list
+ of taskIds that the TaskTracker's TaskMemoryManager manages.
+ (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-5446. Restore TaskTracker metrics. (cdouglas)
+
+ HADOOP-5449. Fixes the history cleaner thread.
+ (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-5479. NameNode should not send empty block replication request to
+ DataNode. (hairong)
+
+ HADOOP-5259. Job with output hdfs:/user/<username>/outputpath (no
+ authority) fails with Wrong FS. (Doug Cutting via hairong)
+
+ HADOOP-5522. Documents the setup/cleanup tasks in the mapred tutorial.
+ (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-5549. ReplicationMonitor should schedule both replication and
+ deletion work in one iteration. (hairong)
+
+ HADOOP-5554. DataNodeCluster and CreateEditsLog should create blocks with
+ the same generation stamp value. (hairong via szetszwo)
+
+ HADOOP-5231. Clones the TaskStatus before passing it to the JobInProgress.
+ (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-4719. Fix documentation of 'ls' format for FsShell. (Ravi Phulari
+ via cdouglas)
+
+ HADOOP-5374. Fixes a NPE problem in getTasksToSave method.
+ (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-4780. Cache the size of directories in DistributedCache, avoiding
+ long delays in recalculating it. (He Yongqiang via cdouglas)
+
+ HADOOP-5551. Prevent directory destruction on file create.
+ (Brian Bockelman via shv)
+
+ HADOOP-5671. Fix FNF exceptions when copying from old versions of
+ HftpFileSystem. (Tsz Wo (Nicholas), SZE via cdouglas)
+
+ HADOOP-5213. Fix Null pointer exception caused when bzip2compression
+ was used and user closed a output stream without writing any data.
+ (Zheng Shao via dhruba)
+
+ HADOOP-5579. Set errno correctly in libhdfs for permission, quota, and FNF
+ conditions. (Brian Bockelman via cdouglas)
+
+ HADOOP-5816. Fixes a problem in the KeyFieldBasedComparator to do with
+ ArrayIndexOutOfBounds exception. (He Yongqiang via ddas)
+
+ HADOOP-5951. Add Apache license header to StorageInfo.java. (Suresh
+ Srinivas via szetszwo)
+
+Release 0.19.1 - 2009-02-23
+
+ IMPROVEMENTS
+
+ HADOOP-4739. Fix spelling and grammar, improve phrasing of some sections in
+ mapred tutorial. (Vivek Ratan via cdouglas)
+
+ HADOOP-3894. DFSClient logging improvements. (Steve Loughran via shv)
+
+ HADOOP-5126. Remove empty file BlocksWithLocations.java (shv)
+
+ HADOOP-5127. Remove public methods in FSDirectory. (Jakob Homan via shv)
+
+ BUG FIXES
+
+ HADOOP-4697. Fix getBlockLocations in KosmosFileSystem to handle multiple
+ blocks correctly. (Sriram Rao via cdouglas)
+
+ HADOOP-4420. Add null checks for job, caused by invalid job IDs.
+ (Aaron Kimball via tomwhite)
+
+ HADOOP-4632. Fix TestJobHistoryVersion to use test.build.dir instead of the
+ current workding directory for scratch space. (Amar Kamat via cdouglas)
+
+ HADOOP-4508. Fix FSDataOutputStream.getPos() for append. (dhruba via
+ szetszwo)
+
+ HADOOP-4727. Fix a group checking bug in fill_stat_structure(...) in
+ fuse-dfs. (Brian Bockelman via szetszwo)
+
+ HADOOP-4836. Correct typos in mapred related documentation. (Jord? Polo
+ via szetszwo)
+
+ HADOOP-4821. Usage description in the Quotas guide documentations are
+ incorrect. (Boris Shkolnik via hairong)
+
+ HADOOP-4847. Moves the loading of OutputCommitter to the Task.
+ (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-4966. Marks completed setup tasks for removal.
+ (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-4982. TestFsck should run in Eclipse. (shv)
+
+ HADOOP-5008. TestReplication#testPendingReplicationRetry leaves an opened
+ fd unclosed. (hairong)
+
+ HADOOP-4906. Fix TaskTracker OOM by keeping a shallow copy of JobConf in
+ TaskTracker.TaskInProgress. (Sharad Agarwal via acmurthy)
+
+ HADOOP-4918. Fix bzip2 compression to work with Sequence Files.
+ (Zheng Shao via dhruba).
+
+ HADOOP-4965. TestFileAppend3 should close FileSystem. (shv)
+
+ HADOOP-4967. Fixes a race condition in the JvmManager to do with killing
+ tasks. (ddas)
+
+ HADOOP-5009. DataNode#shutdown sometimes leaves data block scanner
+ verification log unclosed. (hairong)
+
+ HADOOP-5086. Use the appropriate FileSystem for trash URIs. (cdouglas)
+
+ HADOOP-4955. Make DBOutputFormat us column names from setOutput().
+ (Kevin Peterson via enis)
+
+ HADOOP-4862. Minor : HADOOP-3678 did not remove all the cases of
+ spurious IOExceptions logged by DataNode. (Raghu Angadi)
+
+ HADOOP-5034. NameNode should send both replication and deletion requests
+ to DataNode in one reply to a heartbeat. (hairong)
+
+ HADOOP-4759. Removes temporary output directory for failed and killed
+ tasks by launching special CLEANUP tasks for the same.
+ (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-5161. Accepted sockets do not get placed in
+ DataXceiverServer#childSockets. (hairong)
+
+ HADOOP-5193. Correct calculation of edits modification time. (shv)
+
+ HADOOP-4494. Allow libhdfs to append to files.
+ (Pete Wyckoff via dhruba)
+
+ HADOOP-5166. Fix JobTracker restart to work when ACLs are configured
+ for the JobTracker. (Amar Kamat via yhemanth).
+
+ HADOOP-5067. Fixes TaskInProgress.java to keep track of count of failed and
+ killed tasks correctly. (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-4760. HDFS streams should not throw exceptions when closed twice.
+ (enis)
+
+Release 0.19.0 - 2008-11-18
+
+ INCOMPATIBLE CHANGES
+
+ HADOOP-3595. Remove deprecated methods for mapred.combine.once
+ functionality, which was necessary to providing backwards
+ compatible combiner semantics for 0.18. (cdouglas via omalley)
+
+ HADOOP-3667. Remove the following deprecated methods from JobConf:
+ addInputPath(Path)
+ getInputPaths()
+ getMapOutputCompressionType()
+ getOutputPath()
+ getSystemDir()
+ setInputPath(Path)
+ setMapOutputCompressionType(CompressionType style)
+ setOutputPath(Path)
+ (Amareshwari Sriramadasu via omalley)
+
+ HADOOP-3652. Remove deprecated class OutputFormatBase.
+ (Amareshwari Sriramadasu via cdouglas)
+
+ HADOOP-2885. Break the hadoop.dfs package into separate packages under
+ hadoop.hdfs that reflect whether they are client, server, protocol,
+ etc. DistributedFileSystem and DFSClient have moved and are now
+ considered package private. (Sanjay Radia via omalley)
+
+ HADOOP-2325. Require Java 6. (cutting)
+
+ HADOOP-372. Add support for multiple input paths with a different
+ InputFormat and Mapper for each path. (Chris Smith via tomwhite)
+
+ HADOOP-1700. Support appending to file in HDFS. (dhruba)
+
+ HADOOP-3792. Make FsShell -test consistent with unix semantics, returning
+ zero for true and non-zero for false. (Ben Slusky via cdouglas)
+
+ HADOOP-3664. Remove the deprecated method InputFormat.validateInput,
+ which is no longer needed. (tomwhite via omalley)
+
+ HADOOP-3549. Give more meaningful errno's in libhdfs. In particular,
+ EACCES is returned for permission problems. (Ben Slusky via omalley)
+
+ HADOOP-4036. ResourceStatus was added to TaskTrackerStatus by HADOOP-3759,
+ so increment the InterTrackerProtocol version. (Hemanth Yamijala via
+ omalley)
+
+ HADOOP-3150. Moves task promotion to tasks. Defines a new interface for
+ committing output files. Moves job setup to jobclient, and moves jobcleanup
+ to a separate task. (Amareshwari Sriramadasu via ddas)
+
+ HADOOP-3446. Keep map outputs in memory during the reduce. Remove
+ fs.inmemory.size.mb and replace with properties defining in memory map
+ output retention during the shuffle and reduce relative to maximum heap
+ usage. (cdouglas)
+
+ HADOOP-3245. Adds the feature for supporting JobTracker restart. Running
+ jobs can be recovered from the history file. The history file format has
+ been modified to support recovery. The task attempt ID now has the
+ JobTracker start time to disinguish attempts of the same TIP across
+ restarts. (Amar Ramesh Kamat via ddas)
+
+ HADOOP-4007. REMOVE DFSFileInfo - FileStatus is sufficient.
+ (Sanjay Radia via hairong)
+
+ HADOOP-3722. Fixed Hadoop Streaming and Hadoop Pipes to use the Tool
+ interface and GenericOptionsParser. (Enis Soztutar via acmurthy)
+
+ HADOOP-2816. Cluster summary at name node web reports the space
+ utilization as:
+ Configured Capacity: capacity of all the data directories - Reserved space
+ Present Capacity: Space available for dfs,i.e. remaining+used space
+ DFS Used%: DFS used space/Present Capacity
+ (Suresh Srinivas via hairong)
+
+ HADOOP-3938. Disk space quotas for HDFS. This is similar to namespace
+ quotas in 0.18. (rangadi)
+
+ HADOOP-4293. Make Configuration Writable and remove unreleased
+ WritableJobConf. Configuration.write is renamed to writeXml. (omalley)
+
+ HADOOP-4281. Change dfsadmin to report available disk space in a format
+ consistent with the web interface as defined in HADOOP-2816. (Suresh
+ Srinivas via cdouglas)
+
+ HADOOP-4430. Further change the cluster summary at name node web that was
+ changed in HADOOP-2816:
+ Non DFS Used - This indicates the disk space taken by non DFS file from
+ the Configured capacity
+ DFS Used % - DFS Used % of Configured Capacity
+ DFS Remaining % - Remaing % Configured Capacity available for DFS use
+ DFS command line report reflects the same change. Config parameter
+ dfs.datanode.du.pct is no longer used and is removed from the
+ hadoop-default.xml. (Suresh Srinivas via hairong)
+
+ HADOOP-4116. Balancer should provide better resource management. (hairong)
+
+ HADOOP-4599. BlocksMap and BlockInfo made package private. (shv)
+
+ NEW FEATURES
+
+ HADOOP-3341. Allow streaming jobs to specify the field separator for map
[... 44219 lines stripped ...]