You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-commits@hadoop.apache.org by om...@apache.org on 2011/03/08 06:52:44 UTC
svn commit: r1079179 - in /hadoop/mapreduce/branches/yahoo-merge:
CHANGES.txt HADOOP-CHANGES.txt
Author: omalley
Date: Tue Mar 8 05:52:43 2011
New Revision: 1079179
URL: http://svn.apache.org/viewvc?rev=1079179&view=rev
Log:
commit 23207aad8f2a3aa94937f44f767795f5dda67073
Author: Luke Lu <ll...@yahoo-inc.com>
Date: Fri Oct 29 06:06:14 2010 -0700
Move HADOOP-CHANGES back to CHANGES to ease merges from Apache
Internal changes now go into YAHOO-CHANGES.txt.
+++ b/YAHOO-CHANGES.txt
+Hadoop MapReduce Change Log
+
+Trunk (unreleased changes)
+
+ INCOMPATIBLE CHANGES
+
+ NEW FEATURES
+
+ IMPROVEMENTS
+ Please make it clear why the job failed (Krishna Ramachandran via mahadev) from
+
+ OPTIMIZATIONS
+
+ BUG FIXES
+
Removed:
hadoop/mapreduce/branches/yahoo-merge/HADOOP-CHANGES.txt
Modified:
hadoop/mapreduce/branches/yahoo-merge/CHANGES.txt
Modified: hadoop/mapreduce/branches/yahoo-merge/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/yahoo-merge/CHANGES.txt?rev=1079179&r1=1079178&r2=1079179&view=diff
==============================================================================
--- hadoop/mapreduce/branches/yahoo-merge/CHANGES.txt (original)
+++ hadoop/mapreduce/branches/yahoo-merge/CHANGES.txt Tue Mar 8 05:52:43 2011
@@ -4,12 +4,1855 @@ Trunk (unreleased changes)
INCOMPATIBLE CHANGES
+ MAPREDUCE-1866. Removes deprecated class
+ org.apache.hadoop.streaming.UTF8ByteArrayUtils. (amareshwari)
+
+ MAPREDUCE-1664. Changes the behaviour of the combination of job-acls
+ when they function together with queue-acls. (Ravi Gummadi via vinodkv)
+
NEW FEATURES
+ MAPREDUCE-1804. Stress-test tool for HDFS introduced in HDFS-708.
+ (Joshua Harlow via shv)
+
+ MAPREDUCE-220. Collect cpu and memory statistics per task. (Scott Chen via
+ acmurthy)
+
IMPROVEMENTS
- Bug-2367384 Please make it clear why the job failed (Krishna Ramachandran via mahadev) from http://bug.corp.yahoo.com/attachment.cgi?id=1169543
+
+ MAPREDUCE-1546. Redirect all job pages to corresponding history page
+ if job is not in JT memory. (Scott Chen via sharad)
+
+ MAPREDUCE-1092. Enable assertions for unit tests. (Eli Collins via
+ cdouglas)
+
+ MAPREDUCE-1680. Add a metric recording JobTracker heartbeats processed.
+ (Dick King via cdouglas)
+
+ MAPREDUCE-1761. FairScheduler allows separate configuration of node
+ and rack locality wait time (Scott Chen via dhruba)
+
+ MAPREDUCE-1539. authorization checks for inter-server protocol
+ (based on HADOOP-6600) (Boris Shkolnik via shv)
+
+ MAPREDUCE-1798. Names the configuration keys for the Kerberos
+ principals better. (Boris Shkolnik via ddas)
+
+ MAPREDUCE-1773. streaming doesn't support jobclient.output.filter.
+ (Amareshwari Sriramadasu via vinodkv)
+
+ MAPREDUCE-1785. Add streaming config option for not emitting the key.
+ (Eli Collins via sharad)
+
+ MAPREDUCE-572. If #link is missing from uri format of -cacheArchive
+ then streaming does not throw error. (Amareshwari Sriramadasu via
+ vinodkv)
+
+ MAPREDUCE-1545. Add timestamps for first task type launched in job summary.
+ (Luke Lu via cdouglas)
+
+ MAPREDUCE-1543. Add an audit log for authentication events. (Amar Kamat and
+ Luke Lu via cdouglas)
+
+ MAPREDUCE-1762. Add ability to set values of task counters. (Scott Chen via
+ cdouglas)
+
+ MAPREDUCE-1533. Reduce overhead of logging and string manipulation during
+ heartbeat processing. (Amar Kamat and Dick King via cdouglas)
+
+ MAPREDUCE-1516. JobTracker issues delegation tokens only if the user's
+ authentication is Kerberos. (Jitendra Pandey via ddas)
+
+ MAPREDUCE-647. Update distcp forrest documentation to reflect the changes
+ of HADOOP-5472, MAPREDUCE-642 and HADOOP-5620. (Rodrigo Schmidt via
+ szetszwo)
+
+ MAPREDUCE-1851. Documents configuration parameters in streaming.
+ (amareshwari)
+
+ MAPREDUCE-1868. Add a read and connection timeout to JobClient while
+ pulling tasklogs. (Krishna Ramachandran via acmurthy)
+
+ MAPREDUCE-1788. Ensure failure to setup CompletedJobStatusStore is not
+ silently ignored by the JobTracker. (Krishna Ramachandran via acmurthy)
+
+ MAPREDUCE-1850. Includes job submit host information (name and ip) in
+ jobconf and jobdetails display (Krishna Ramachandran via amareshwari)
+
+ MAPREDUCE-1893. Slive with multiple reducers. (shv)
+
+ MAPREDUCE-1248. Fixes redudant memory copying in StreamKeyValUtil.
+ (Ruibang He via amareshwari)
+
+ MAPREDUCE-1840. Enhancements to Gridmix benchmark simulating user
+ diversity, queue replay, and task duration for JobTracker load testing.
+ Also includes compatibility with security enhancements, and scalability
+ improvements. (Amar Kamat, Rahul Singh, Hong Tang, and cdouglas)
+
+ MAPREDUCE-1848. Put number of speculative, data local, rack local
+ tasks in JobTracker metrics. (Scott Chen via dhruba)
+
+ MAPREDUCE-1935. Makes the Distcp to work in a secure environment.
+ (Boris Shkolnik via ddas)
+
+ MAPREDUCE-1945. The MapReduce component for HADOOP-6632.
+ (Kan Zhang & Jitendra Pandey via ddas)
+
+ MAPREDUCE-1936. Modify Gridmix3 to support more tunable parameters for
+ stress submission and sleep jobs. (Hong Tang via cdouglas)
+
+ MAPREDUCE-1733. Makes pipes applications secure. (Jitendra Pandey via ddas)
+
+ MAPREDUCE-1566. Adds a configuration attribute using which job clients can
+ specify a credentials file. The tokens from there will be passed to the job.
+ (Jitendra Pandey and Owen O'Malley via ddas)
+
+ MAPREDUCE-1624. Documents the job credentials and associated details to do
+ with delegation tokens (on the client side).
+ (Jitendra Pandey and Devaraj Das via ddas)
+
+ MAPREDUCE-1834. TestSimulatorDeterministicReplay timesout on trunk.
+ (Hong Tang via mahadev)
+
+ MAPREDUCE-1993. Fixes test failure
+ TestTrackerDistributedCacheManagerWithLinuxTaskController. (Devaraj Das
+ via amareshwari)
+
+ MAPREDUCE-1523. Making Mumak work with Capacity-Scheduler (Anirban Das
+ via mahadev)
+
+ MAPREDUCE-1920. Enables completed jobstatus store by default. (Tom White
+ via amareshwari)
+
+ MAPREDUCE-1881. Improve TaskTrackerInstrumentation to enable collection of
+ advanced metrics. (Matei Zaharia via acmurthy)
+
+ MAPREDUCE-1548. Hadoop archives preserve times and other properties from
+ original files. (Rodrigo Schmidt via dhruba)
+
+ MAPREDUCE-1517. Supports streaming job to run in the background. (Bochun Bai
+ via amareshwari)
+
+ MAPREDUCE-1819. RaidNode is now smarter in submitting Raid jobs. (Ramkumar
+ Vadali via schen)
OPTIMIZATIONS
+ MAPREDUCE-1354. Enhancements to JobTracker for better performance and
+ scalability. (Arun C. Murthy & Richard King via acmurthy)
+
+ MAPREDUCE-1829. JobInProgress.findSpeculativeTask should use min() to
+ find the candidate instead of sort(). (Scott Chen via vinodkv)
+
+ BUG FIXES
+
+ MAPREDUCE-1845. FairScheduler.tasksToPreempt() can return negative number.
+ (Scott Chen via matei)
+
+ MAPREDUCE-1707. TaskRunner can get NPE in getting ugi from TaskTracker.
+ (Vinod Kumar Vavilapalli)
+
+ MAPREDUCE-1532. Ensures that delegation tokens is obtained as the
+ actual user when the proxy-user is used for submitting jobs. Also
+ refactors the DelegationTokenToRenew class. (ddas)
+
+ MAPREDUCE-1558. Fixes MRAdmin to look up the principal of the
+ JobTracker and use that in the RefreshUserToGroupsMapping protocol and
+ RefreshAuthorizationPolicyProtocol. (Boris Shkolnik via ddas)
+
+ MAPREDUCE-1662. Remove unused methods from TaskRunner. (Amareshwari
+ Sriramadasu via cdouglas)
+
+ MAPREDUCE-1617. Use IPv4 stack for unit tests. (Amar Kamat and Luke Lu via
+ cdouglas)
+
+ MAPREDUCE-1599. Fixes MRBench so that it reuses tokens across jobs
+ correctly. (Jitendra Nath Pandey via ddas)
+
+ MAPREDUCE-1836. Refresh for proxy superuser config (mr part for HDFS-1096).
+ (Boris Shkolnik via shv)
+
+ MAPREDUCE-1505. Create RPC client on job submission, not in cstr of Job
+ instance. (Dick King via cdouglas)
+
+ MAPREDUCE-1813. NPE in PipeMapred.MRErrorThread. (Ravi Gummadi via vinodkv)
+
+ MAPREDUCE-1225. Fixes DistributedCache to check if the file is fresh or not,
+ for the first localization also. (Zhong Wang via amareshwari)
+
+ MAPREDUCE-1559. Fixes the token renewer to use the JobTracker's
+ credentials for talking to the NameNode. (ddas)
+
+ MAPREDUCE-1492. Delete obsolete har files used on the parity files
+ of hdfs raid. (Rodrigo Schmidt via dhruba)
+
+ MAPREDUCE-1857. Removes unused configuration parameters in streaming.
+ (amareshwari)
+
+ MAPREDUCE-1887. MRAsyncDiskService now properly absolutizes volume root
+ paths. (Aaron Kimball via zshao)
+
+ MAPREDUCE-1863. Fix NPE in Rumen when processing null CDF for failed task
+ attempts. (Amar Kamat via cdouglas)
+
+ MAPREDUCE-1864. Removes uninitialized/unused variables in
+ org.apache.hadoop.streaming.PipeMapRed. (amareshwari)
+
+ MAPREDUCE-1888. Fixes Streaming to override output key and value types,
+ only if mapper/reducer is a command. (Ravi Gummadi via amareshwari)
+
+ MAPREDUCE-577. Fixes duplicate records in StreamXmlRecordReader.
+ (Ravi Gummadi via amareshwari)
+
+ MAPREDUCE-1894. Fixed a bug in DistributedRaidFileSystem.readFully()
+ that was causing it to loop infinitely. (Ramkumar Vadali via dhruba)
+
+ MAPREDUCE-1838. Reduce the time needed for raiding a bunch of files
+ by randomly assigning files to map tasks. (Ramkumar Vadali via dhruba)
+
+ MAPREDUCE-1820. Fix InputSampler to clone sampled keys. (Alex Kozlov via
+ cdouglas)
+
+ MAPREDUCE-1528. Incorporates the changes to the credentials API done in
+ HADOOP-6845. Also, introduces Credentials in JobConf, and in JobContext.
+ (Jitendra Pandey and Arun Murthy via ddas)
+
+ MAPREDUCE-1865. Rumen should also support jobhistory files generated using
+ trunk. (Amar Kamat via amareshwari)
+
+ MAPREDUCE-1621. Fixes NPE in TextOutputReader.getLastOutput if it has never
+ read any output. (amareshwari)
+
+ MAPREDUCE-1911. Fixes errors in -info message in streaming. (amareshwari)
+
+ MAPREDUCE-1772. Corrects errors in streaming documentation in forrest.
+ (amareshwari)
+
+ MAPREDUCE-1925. Fix failing TestRumenJobTraces. (Ravi Gummadi via cdouglas)
+
+ MAPREDUCE-1718. Fixes a bug in the construction of jobconf key for the
+ mapping that the tasks use at runtime for looking up delegation tokens.
+ (Boris Shkolnik via ddas)
+
+ MAPREDUCE-1701. Fixes a problem to do with exception handling in
+ delegation-token renewals. (Boris Shkolnik via ddas)
+
+ MAPREDUCE-1686. Fixes StreamUtil.goodClassOrNull to find classes without
+ package names. (Paul Burkhardt via amareshwari)
+
+ MAPREDUCE-1288. Fixes TrackerDistributedCacheManager to take into account
+ the owner of the localized file in the mapping from cache URIs to
+ CacheStatus objects. (ddas)
+
+ MAPREDUCE-1982. Fixes Rumen's TraceBuilder to extract job name from either
+ of configuration properties "mapreduce.job.name" and "mapred.job.name".
+ (Ravi Gummadi via amareshwari)
+
+ MAPREDUCE-1958. The MapReduce part corresponding to the HADOOP-6873.
+ (Boris Shkolnik & Owen O'Malley via ddas)
+
+ MAPREDUCE-1900. TaskTracker and JobTracker closes FileSystems, opened on
+ behalf of users that it no longer requires. (Kan Zhang and ddas via ddas)
+
+ MAPREDUCE-1992. Fixes a problem to do with bringing up the JobTracker in
+ unsecure mode. (Kan Zhang via ddas)
+
+ MAPREDUCE-1999. Fixes ClientProtocol to use the correct
+ DelegationTokenSelector. (Jitendra Pandey via ddas)
+
+ MAPREDUCE-1780. AccessControlList.toString() is used for serialization of
+ ACL in JobStatus.java. (Ravi Gummadi via vinodkv)
+
+ MAPREDUCE-1961. Fix ConcurrentModificationException in Gridmix during
+ shutdown. (Hong Tang via cdouglas)
+
+ MAPREDUCE-2000. Fix parsing of JobHistory lines in Rumen when quotes are
+ escaped. (Hong Tang via cdouglas)
+
+ MAPREDUCE-2022. Fixes compilation errors in TestSubmitJob. (amareshwari)
+
+ MAPREDUCE-1670. RAID policies should not scan their own destination path.
+ (Ramkumar Vadali via dhruba)
+
+ MAPREDUCE-1668. RaidNode Hars a directory only if all its parity files
+ have been created. (Ramkumar Vadali via dhruba)
+
+ MAPREDUCE-2021. Fixes duplicate hostnames in CombineFileInputFormat's
+ split locations. (amareshwari)
+
+ MAPREDUCE-1375. Fixes flaky test TestFileArgs. (Todd Lipcon via
+ amareshwari)
+
+ MAPREDUCE-2023. TestDFSIO should not stop reading if curSize != bufferSize.
+ (Hong Tang via szetszwo)
+
+ MAPREDUCE-2031. Fixes test failures TestTaskLauncher and
+ TestTaskTrackerLocalization. (Ravi Gummadi via amareshwari)
+
+ MAPREDUCE-2046. Fixes CombineFileInputFormat to allow splits with size
+ less than DFS block size. (dhruba borthakur via amareshwari)
+
+ MAPREDUCE-1975. Fixes unnecessary InterruptedException log in gridmix.
+ (Ravi Gummadi via amareshwari)
+
+ MAPREDUCE-1597. Fixes CombineFileInputFormat to work with non-splittable
+ files. (amareshwari)
+
+ MAPREDUCE-2032. Fixes TestJobCleanup to cleanup test directory in
+ tearDown. (Dick King via amareshwari)
+
+ MAPREDUCE-1979. Fixes "Output directory already exists" error in gridmix
+ when gridmix.output.directory is not defined. (Ravi Gummadi via
+ amareshwari)
+
+ MAPREDUCE-1918. Adds documentation to Rumen. (Amar Kamat via amareshwari)
+
+ MAPREDUCE-2078. Fixes TraceBuilder to generate traces when a globbed job
+ history path is given. (Amar Kamat via amareshwari)
+
+ MAPREDUCE-1989. Fixes error message in gridmix when user resolver is set
+ and no user list is given. (Ravi Gummadi via amareshwari)
+
+ MAPREDUCE-2067. Distinct minicluster services (e.g. NN and JT) overwrite
+ each other's service policies. (Aaron T. Myers via tomwhite)
+
+ MAPREDUCE-2029. DistributedRaidFileSystem removes itself from FileSystem
+ cache when it is closed. (Ramkumar Vadali via dhruba)
+
+ MAPREDUCE-1816. HAR files used for RAID parity-bite have configurable
+ partfile size. (Ramkumar Vadali via dhruba)
+
+ MAPREDUCE-2082. Fixes Pipes to create the jobtoken file in the right
+ place. (Jitendra Pandey via ddas)
+
+ MAPREDUCE-2095. Fixes Gridmix to run from compressed traces. (Ranjit
+ Mathew via amareshwari)
+
+ MAPREDUCE-1908. DistributedRaidFileSystem now handles ChecksumException
+ correctly. (Ramkumar Vadali via schen)
+
+Release 0.21.1 - Unreleased
+
+ NEW FEATURES
+
+ MAPREDUCE-2040. Forrest Documentation for Dynamic Priority Scheduler.
+ (Thomas Sandholm via tomwhite)
+
BUG FIXES
+ MAPREDUCE-1897. trunk build broken on compile-mapred-test (cos)
+
+ MAPREDUCE-1280. Update Eclipse plugin to the new eclipse.jdt API.
+ (Alex Kozlov via szetszwo)
+
+ MAPREDUCE-1984. herriot TestCluster fails because exclusion is not there
+ (Balaji Rajagopalan via cos)
+
+ MAPREDUCE-2090. Clover build doesn't generate per-test coverage. (cos)
+
+Release 0.21.0 - 2010-08-13
+
+ INCOMPATIBLE CHANGES
+
+ MAPREDUCE-516. Fix the starvation problem in the Capacity Scheduler
+ when running High RAM Jobs. (Arun Murthy via yhemanth)
+
+ MAPREDUCE-358. Change org.apache.hadoop.examples. AggregateWordCount
+ and org.apache.hadoop.examples.AggregateWordHistogram to use new
+ mapreduce api. (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-245. Change Job and jobcontrol classes to use the List interface
+ rather than ArrayList in APIs. (Tom White via cdouglas)
+
+ MAPREDUCE-766. Enhanced list-blacklisted-trackers to display reasons
+ for blacklisting a node. (Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-817. Add a cache for retired jobs with minimal job info and
+ provide a way to access history file url. (sharad)
+
+ MAPREDUCE-711. Moved Distributed Cache from Common to Map/Reduce
+ project. (Vinod Kumar Vavilapalli via yhemanth)
+
+ MAPREDUCE-895. Per the contract elucidated in HADOOP-6201, throw
+ FileNotFoundException from FileSystem::listStatus rather than returning
+ null. (Jakob Homan via cdouglas)
+
+ MAPREDUCE-479. Provide full task id to map output servlet rather than the
+ reduce id, only. (Jiaqi Tan via cdouglas)
+
+ MAPREDUCE-873. Simplify job recovery. Incomplete jobs are resubmitted on
+ jobtracker restart. Removes a public constructor in JobInProgress. (sharad)
+
+ HADOOP-6230. Moved process tree and memory calculator related classes from
+ Common to Map/Reduce. (Vinod Kumar Vavilapalli via yhemanth)
+
+ MAPREDUCE-157. Refactor job history APIs and change the history format to
+ JSON. (Jothi Padmanabhan via sharad)
+
+ MAPREDUCE-849. Rename configuration properties. (Amareshwari Sriramadasu
+ via sharad)
+
+ MAPREDUCE-1287. Only call the partitioner with more than one reducer.
+ (cdouglas)
+
+ MAPREDUCE-1385. Use the new UserGroupInformation from HADOOP-6299.
+ (ddas via omalley)
+
+ MAPREDUCE-1493. Authorization for job-history pages. (vinodkv)
+
+ MAPREDUCE-1607. Task controller may not set permissions for a
+ task cleanup attempt's log directory (Amareshwari Sriramadasu via vinodkv)
+
+ MAPREDUCE-1683. Remove JNI calls from ClusterStatus cstr. (Arun Murthy and
+ Luke Lu via cdouglas)
+
+ MAPREDUCE-1855. Makes the refresh methods (for groups and proxy users)
+ independent of the client side configuration. (Boris Shkolnik via ddas)
+
+ NEW FEATURES
+
+ MAPREDUCE-1774. Large-scale Automated Framework (Sharad Agarwal, Sreekanth
+ Ramakrishnan, Konstantin Boudnik, at all via cos)
+
+ MAPREDUCE-706. Support for FIFO pools in the fair scheduler.
+ (Matei Zaharia)
+
+ MAPREDUCE-546. Provide sample fair scheduler config file in conf/ and use
+ it by default if no other config file is specified. (Matei Zaharia)
+
+ MAPREDUCE-551. Preemption support in the Fair Scheduler. (Matei Zaharia)
+
+ MAPREDUCE-567. Add a new example MR that always fails. (Philip Zeyliger
+ via tomwhite)
+
+ MAPREDUCE-211. Provides ability to run a health check script on the
+ tasktracker nodes and blacklist nodes if they are unhealthy.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-637. Add an example, distbbp, which able to compute the n th bit
+ of Pi for some large n. (szetszwo)
+
+ MAPREDUCE-532. Provide a way to limit the number of used slots
+ per queue in the capacity scheduler.
+ (Rahul Kumar Singh via yhemanth)
+
+ MAPREDUCE-467. Provide ability to collect statistics about total tasks
+ and succeeded tasks in different time windows. (sharad)
+
+ MAPREDUCE-740. Log a job-summary at the end of a job, while allowing it
+ to be configured to use a custom appender if desired. (acmurthy)
+
+ MAPREDUCE-814. Provide a way to configure completed job history files
+ to be on HDFS. (sharad)
+
+ MAPREDUCE-800. MRUnit should support the new API. (Aaron Kimball via
+ tomwhite)
+
+ MAPREDUCE-798. MRUnit should be able to test a succession of MapReduce
+ passes. (Aaron Kimball via tomwhite)
+
+ MAPREDUCE-768. Provide an option to dump jobtracker configuration in JSON
+ format to standard output. (V.V.Chaitanya Krishna via yhemanth)
+
+ MAPREDUCE-824. Add support for a hierarchy of queues in the capacity
+ scheduler. (Rahul Kumar Singh via yhemanth)
+
+ MAPREDUCE-751. Add Rumen, a tool for extracting statistics from job tracker
+ logs and generating job traces for simulation and analysis. (Dick King via
+ cdouglas)
+
+ MAPREDUCE-830. Add support for splittable compression to TextInputFormats.
+ (Abdul Qadeer via cdouglas)
+
+ MAPREDUCE-861. Add support for hierarchical queues in the Map/Reduce
+ framework. (Rahul Kumar Singh via yhemanth)
+
+ MAPREDUCE-776. Add Gridmix, a benchmark processing Rumen traces to simulate
+ a measured mix of jobs on a cluster. (cdouglas)
+
+ MAPREDUCE-862. Enhance JobTracker UI to display hierarchical queues.
+ (V.V.Chaitanya Krishna via yhemanth)
+
+ MAPREDUCE-777. Brand new apis to track and query jobs as a
+ replacement for JobClient. (Amareshwari Sriramadasu via acmurthy)
+
+ MAPREDUCE-775. Add native and streaming support for Vertica as an input
+ or output format taking advantage of parallel read and write properties of
+ the DBMS. (Omer Trajman via ddas)
+
+ MAPREDUCE-679. XML-based metrics as JSP servlet for JobTracker.
+ (Aaron Kimball via tomwhite)
+
+ MAPREDUCE-980. Modify JobHistory to use Avro for serialization. (cutting)
+
+ MAPREDUCE-728. Add Mumak, a Hadoop map/reduce simulator. (Arun C Murthy,
+ Tamas Sarlos, Anirban Dasgupta, Guanying Wang, and Hong Tang via cdouglas)
+
+ MAPREDUCE-1383. Automates fetching of delegation tokens in File*Formats
+ Distributed Cache and Distcp. Also, provides a config
+ mapreduce.job.hdfs-servers that the jobs can populate with a comma
+ separated list of namenodes. The job client automatically fetches
+ delegation tokens from those namenodes. (Boris Shkolnik via ddas)
+
+ MAPREDUCE-698. Per-pool task limits for the fair scheduler.
+ (Kevin Peterson via matei)
+
+ MAPREDUCE-1026. Does mutual authentication of the shuffle
+ transfers using a shared JobTracker generated key.
+ (Boris Shkolnik via ddas)
+
+ MAPREDUCE-744. Introduces the notion of a public distributed cache.
+ (Devaraj Das)
+
+ MAPREDUCE-1338. Introduces the notion of token cache using which
+ tokens and secrets can be sent by the Job client to the JobTracker.
+ (Boris Shkolnik via ddas)
+
+ HDFS-503. This patch implements an optional layer over HDFS that
+ implements offline erasure-coding. It can be used to reduce the
+ total storage requirements of HDFS. (dhruba)
+
+ MAPREDUCE-1432. Adds hooks in the jobtracker and tasktracker
+ for loading the tokens in the user's ugi. This is required
+ for the copying of files from the hdfs. (ddas)
+
+ MAPREDUCE-1335. Adds SASL Kerberos/Digest authentication in MapReduce.
+ (Kan Zhang via ddas)
+
+ MAPREDUCE-1464. Makes a compatible change in JobTokenIdentifier to
+ account for HADOOP-6510. (Jitendra Nath Pandey via ddas)
+
+ MAPREDUCE-1433. Add a delegation token for MapReduce. (omalley)
+
+ MAPREDUCE-1307. Introduces the Job level ACLs feature.
+ (Vinod Kumar Vavilapalli via ddas)
+
+ MAPREDUCE-1430. JobTracker automatically renews delegation tokens for jobs.
+ (Boris Shkolnik via ddas)
+
+ MAPREDUCE-1455. Introduces job-level authorization for mapreduce servlets.
+ (Ravi Gummadi via vinodkv)
+
+ IMPROVEMENTS
+
+ MAPREDUCE-463. Makes job setup and cleanup tasks as optional.
+ (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-502. Allow jobtracker to be configured with zero completed jobs
+ in memory. (Amar Kamat via sharad)
+
+ MAPREDUCE-416. Moves the history file to a "done" folder whenever a job
+ completes. (Amar Kamat via ddas)
+
+ MAPREDUCE-646. Increase srcfilelist replication number in dictcp job.
+ (Ravi Gummadi via szetszwo)
+
+ HADOOP-6106. Updated hadoop-core and test jars from hudson trunk
+ build #12. (Giridharan Kesavan)
+
+ MAPREDUCE-642. A option to distcp that allows preserving the full
+ source path of a file in the specified destination directory.
+ (Rodrigo Schmidt via dhruba)
+
+ MAPREDUCE-686. Move TestSpeculativeExecution.Fake* into a separate class
+ so that it can be used by other tests. (Jothi Padmanabhan via sharad)
+
+ MAPREDUCE-625. Modify TestTaskLimits to improve execution time.
+ (Jothi Padmanabhan via sharad)
+
+ MAPREDUCE-465. Deprecate o.a.h.mapred.lib.MultithreadedMapRunner and add
+ test for o.a.h.mapreduce.lib.MultithreadedMapper.
+ (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-701. Improves the runtime of the TestRackAwareTaskPlacement
+ by making it a unit test. (Jothi Padmanabhan via ddas)
+
+ MAPREDUCE-371. Change KeyFieldBasedComparator and KeyFieldBasedPartitioner
+ to use new api. (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-623. Resolve javac warnings in mapreduce. (Jothi Padmanabhan
+ via sharad)
+
+ MAPREDUCE-655. Change KeyValueLineRecordReader and KeyValueTextInputFormat
+ to use new mapreduce api. (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-632. Merge TestCustomOutputCommitter with
+ TestCommandLineJobSubmission. (Jothi Padmanabhan via sharad)
+
+ MAPREDUCE-627. Improves execution time of TestTrackerBlacklistAcrossJobs.
+ (Jothi Padmanabhan via ddas)
+
+ MAPREDUCE-630. Improves execution time of TestKillCompletedJob.
+ (Jothi Padmanabhan via ddas)
+
+ MAPREDUCE-626. Improves the execution time of TestLostTracker.
+ (Jothi Padmanabhan via ddas)
+
+ MAPREDUCE-353. Makes the shuffle read and connection timeouts
+ configurable. (Ravi Gummadi via ddas)
+
+ MAPREDUCE-739. Allow relative paths to be created in archives. (Mahadev
+ Konar via cdouglas)
+
+ MAPREDUCE-772. Merge HADOOP-4010 changes to LineRecordReader into mapreduce
+ package. (Abdul Qadeer via cdouglas)
+
+ MAPREDUCE-785. Separate sub-test of TestReduceFetch to be included in
+ MR-670. (Jothi Padmanabhan via cdouglas)
+
+ MAPREDUCE-784. Modify TestUserDefinedCounters to use LocalJobRunner
+ instead of MiniMR. (Jothi Padmanabhan via sharad)
+
+ HADOOP-6160. Fix releaseaudit target to run on specific directories.
+ (gkesavan)
+
+ MAPREDUCE-782. Use PureJavaCrc32 in SpillRecord. (Todd Lipcon via
+ szetszwo)
+
+ MAPREDUCE-369. Change org.apache.hadoop.mapred.lib.MultipleInputs to
+ use new api. (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-373. Change org.apache.hadoop.mapred.lib.FieldSelectionMapReduce
+ to use new api. (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-628. Improves the execution time of TestJobInProgress.
+ (Jothi Padmanabhan via ddas)
+
+ MAPREDUCE-793. Creates a new test that consolidates a few tests to
+ include in the commit-test list. (Jothi Padmanabhan via ddas)
+
+ MAPREDUCE-797. Adds combiner support to MRUnit MapReduceDriver.
+ (Aaron Kimball via johan)
+
+ MAPREDUCE-656. Change org.apache.hadoop.mapred.SequenceFile* classes
+ to use new mapreduce api. (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-670. Creates ant target for 10 mins patch test build.
+ (Jothi Padmanabhan via gkesavan)
+
+ MAPREDUCE-375. Change org.apache.hadoop.mapred.lib.NLineInputFormat
+ and org.apache.hadoop.mapred.MapFileOutputFormat to use new api.
+ (Amareshwari Sriramadasu via ddas)
+
+ MAPREDUCE-779. Added node health failure counts into
+ JobTrackerStatistics. (Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-842. Setup secure permissions for localized job files,
+ intermediate outputs and log files on tasktrackers.
+ (Vinod Kumar Vavilapalli via yhemanth)
+
+ MAPREDUCE-478. Allow map and reduce jvm parameters, environment variables
+ and ulimit to be set separately.
+ Configuration changes:
+ add mapred.map.child.java.opts
+ add mapred.reduce.child.java.opts
+ add mapred.map.child.env
+ add mapred.reduce.child.ulimit
+ add mapred.map.child.env
+ add mapred.reduce.child.ulimit
+ deprecated mapred.child.java.opts
+ deprecated mapred.child.env
+ deprecated mapred.child.ulimit
+ (acmurthy)
+
+ MAPREDUCE-767. Remove the dependence on the CLI 2.0 snapshot.
+ (Amar Kamat via omalley)
+
+ MAPREDUCE-712. Minor efficiency tweaks to RandomTextWriter. (cdouglas)
+
+ MAPREDUCE-870. Remove the job retire thread and the associated
+ config parameters. (sharad)
+
+ MAPREDUCE-874. Rename the PiEstimator example to QuasiMonteCarlo.
+ (szetszwo)
+
+ MAPREDUCE-336. Allow logging level of map/reduce tasks to be configurable.
+ Configuration changes:
+ add mapred.map.child.log.level
+ add mapred.reduce.child.log.level
+ (acmurthy)
+
+ MAPREDUCE-355. Update mapred.join package to use the new API. (Amareshwari
+ Sriramadasu via cdouglas)
+
+ HADOOP-6184. Updated hadoop common and test jars to get the new API
+ in Configuration for dumping in JSON format from Hudson trunk build #68.
+ (yhemanth)
+
+ MAPREDUCE-476. Extend DistributedCache to work locally (LocalJobRunner).
+ (Philip Zeyliger via tomwhite)
+
+ MAPREDUCE-825. JobClient completion poll interval of 5s causes slow tests
+ in local mode. (Aaron Kimball via tomwhite)
+
+ MAPREDUCE-910. Support counters in MRUnit. (Aaron Kimball via cdouglas)
+
+ MAPREDUCE-788. Update gridmix2 to use the new API (Amareshwari Sriramadasu
+ via cdouglas)
+
+ MAPREDUCE-875. Make DBRecordReader execute queries lazily. (Aaron Kimball
+ via enis)
+
+ MAPREDUCE-318. Modularizes the shuffle code. (Jothi Padmanabhan and
+ Arun Murthy via ddas)
+
+ MAPREDUCE-936. Allow a load difference for fairshare scheduler.
+ (Zheng Shao via dhruba)
+
+ MAPREDUCE-370. Update MultipleOutputs to use the API, merge funcitonality
+ of MultipleOutputFormat. (Amareshwari Sriramadasu via cdouglas)
+
+ MAPREDUCE-898. Changes DistributedCache to use the new API.
+ (Amareshwari Sriramadasu via ddas)
+
+ MAPREDUCE-144. Includes dump of the process tree in task diagnostics when
+ a task is killed due to exceeding memory limits.
+ (Vinod Kumar Vavilapalli via yhemanth)
+
+ MAPREDUCE-945. Modifies MRBench and TestMapRed to use ToolRunner so that
+ options such as queue name can be passed via command line.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-963. Deprecate o.a.h.mapred.FileAlreadyExistsException and
+ replace it with o.a.h.fs.FileAlreadyExistsException. (Boris Shkolnik
+ via szetszwo)
+
+ MAPREDUCE-960. Remove an unnecessary intermediate copy and obsolete API
+ from KeyValueLineRecordReader. (cdouglas)
+
+ MAPREDUCE-930. Modify Rumen to resolve paths in the canonical way, rather
+ than defaulting to the local filesystem. (cdouglas)
+
+ MAPREDUCE-944. Extend the LoadManager API of the fair-share scheduler
+ to support regulating tasks for a job based on resources currently in use
+ by that job. (dhruba)
+
+ MAPREDUCE-973. Move FailJob and SleepJob from examples to test. (cdouglas
+ via omalley)
+
+ MAPREDUCE-966. Modify Rumen to clean up interfaces and simplify integration
+ with other tools. (Hong Tang via cdouglas)
+
+ MAPREDUCE-856. Setup secure permissions for distributed cache files.
+ (Vinod Kumar Vavilapalli via yhemanth)
+
+ MAPREDUCE-885. More efficient SQL queries for DBInputFormat. (Aaron Kimball
+ via enis)
+
+ MAPREDUCE-284. Enables ipc.client.tcpnodelay in Tasktracker's Child.
+ (Ravi Gummadi via sharad)
+
+ MAPREDUCE-916. Split the documentation to match the project split.
+ (Corinne Chandel via omalley)
+
+ MAPREDUCE-649. Validate a copy by comparing the source and destination
+ checksums in distcp. Also adds an intra-task retry mechanism for errors
+ detected during the copy. (Ravi Gummadi via cdouglas)
+
+ MAPREDUCE-654. Add a -dryrun option to distcp printing a summary of the
+ file data to be copied, without actually performing the copy. (Ravi Gummadi
+ via cdouglas)
+
+ MAPREDUCE-664. Display the number of files deleted by distcp when the
+ -delete option is specified. (Ravi Gummadi via cdouglas)
+
+ MAPREDUCE-781. Let the name of distcp jobs be configurable. (Venkatesh S
+ via cdouglas)
+
+ MAPREDUCE-975. Add an API in job client to get the history file url for
+ a given job id. (sharad)
+
+ MAPREDUCE-905. Add Eclipse launch tasks for MapReduce. (Philip Zeyliger
+ via tomwhite)
+
+ MAPREDUCE-277. Makes job history counters available on the job history
+ viewers. (Jothi Padmanabhan via ddas)
+
+ MAPREDUCE-893. Provides an ability to refresh queue configuration
+ without restarting the JobTracker.
+ (Vinod Kumar Vavilapalli and Rahul Kumar Singh via yhemanth)
+
+ MAPREDUCE-1011. Add build.properties to svn and git ignore. (omalley)
+
+ MAPREDUCE-954. Change Map-Reduce context objects to be interfaces.
+ (acmurthy)
+
+ MAPREDUCE-639. Change Terasort example to reflect the 2009 updates.
+ (omalley)
+
+ MAPREDUCE-1063. Document gridmix benchmark. (cdouglas)
+
+ MAPREDUCE-931. Use built-in interpolation classes for making up task
+ runtimes in Rumen. (Dick King via cdouglas)
+
+ MAPREDUCE-1012. Mark Context interfaces as public evolving. (Tom White via
+ cdouglas)
+
+ MAPREDUCE-971. Document use of distcp when copying to s3, managing timeouts
+ in particular. (Aaron Kimball via cdouglas)
+
+ HDFS-663. DFSIO for append. (shv)
+
+ HDFS-641. Move all of the components that depend on map/reduce to
+ map/reduce. (omalley)
+
+ HADOOP-5107. Use Maven ant tasks to publish artifacts. (Giridharan Kesavan
+ via omalley)
+
+ MAPREDUCE-1229. Allow customization of job submission policy in Mumak.
+ (Hong Tang via cdouglas)
+
+ MAPREDUCE-1317. Reduce the memory footprint of Rumen objects by interning
+ host Strings. (Hong Tang via cdouglas)
+
+ MAPREDUCE-1097. Add support for Vertica 3.5 to its contrib module. (Omer
+ Trajman via cdouglas)
+
+ MAPREDUCE-1627. HadoopArchives should not uses a method in DistCp.
+ (szetszwo)
+
+ MAPREDUCE-1198. Alternatively schedule different types of tasks in
+ fair share scheduler. (Scott Chen via matei)
+
+ MAPREDUCE-707. Provide a jobconf property for explicitly assigning a job to
+ a pool in the Fair Scheduler. (Alan Heirich via matei)
+
+ MAPREDUCE-947. Added commitJob and abortJob apis to OutputCommitter.
+ Enhanced FileOutputCommitter to create a _SUCCESS file for successful
+ jobs. (Amar Kamat & Jothi Padmanabhan via acmurthy)
+
+ MAPREDUCE-1103. Added more metrics to Jobtracker. (sharad)
+
+ MAPREDUCE-1048. Add occupied/reserved slot usage summary on jobtracker UI.
+ (Amareshwari Sriramadasu and Hemanth Yamijala via sharad)
+
+ MAPREDUCE-1090. Modified log statement in TaskMemoryManagerThread to
+ include task attempt id. (yhemanth)
+
+ MAPREDUCE-1189. Reduce ivy console output to ovservable level (cos)
+
+ MAPREDUCE-1167. ProcfsBasedProcessTree collects rss memory information.
+ (Scott Chen via dhruba)
+
+ MAPREDUCE-1231. Added a new DistCp option, -skipcrccheck, so that the CRC
+ check during setup can be skipped. (Jothi Padmanabhan via szetszwo)
+
+ MAPREDUCE-1190. Add package documentation for BBP example.
+ (Tsz Wo (Nicholas) Sze via cdouglas)
+
+ MAPREDUCE-1119. When tasks fail to report status, show tasks's stack dump
+ before killing. (Aaron Kimball via tomwhite)
+
+ MAPREDUCE-1185. Redirect running job url to history url if job is already
+ retired. (Amareshwari Sriramadasu and Sharad Agarwal via sharad)
+
+ MAPREDUCE-1050. Introduce a mock object testing framework. (tomwhite)
+
+ MAPREDUCE-1084. Implementing aspects development and fault injeciton
+ framework for MapReduce. (Sreekanth Ramakrishnan via cos)
+
+ MAPREDUCE-1209. Move common specific part of the test TestReflectionUtils
+ out of mapred into common. (Todd Lipcon via tomwhite)
+
+ MAPREDUCE-967. TaskTracker does not need to fully unjar job jars.
+ (Todd Lipcon via tomwhite)
+
+ MAPREDUCE-1083. Changes in MapReduce so that group information of users
+ can be refreshed in the JobTracker via command line.
+ (Boris Shkolnik via ddas)
+
+ MAPREDUCE-181. Changes the job submission process to be secure.
+ (Devaraj Das)
+
+ MAPREDUCE-1250. Refactors the JobToken to use Common's Token interface.
+ (Kan Zhang via ddas)
+
+ MAPREDUCE-896. Enhance tasktracker to cleanup files that might have
+ been created by user tasks with non-writable permissions.
+ (Ravi Gummadi via yhemanth)
+
+ MAPREDUCE-372. Change org.apache.hadoop.mapred.lib.ChainMapper/Reducer
+ to use new mapreduce api. (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-1295. Add a tool in Rumen for folding and manipulating job
+ traces. (Dick King via cdouglas)
+
+ MAPREDUCE-1302. TrackerDistributedCacheManager deletes file
+ asynchronously, thus reducing task initialization delays.
+ (Zheng Shao via dhruba)
+
+ MAPREDUCE-1218. TaskTrackers send cpu and memory usage of
+ node to JobTracker. (Scott Chen via dhruba)
+
+ MAPREDUCE-847. Fix Releaseaudit warning count to zero
+ (Giridharan Kesavan)
+
+ MAPREDUCE-1337. Use generics in StreamJob to improve readability of that
+ class. (Kay Kay via cdouglas)
+
+ MAPREDUCE-361. Port terasort example to the new mapreduce API. (Amareshwari
+ Sriramadasu via cdouglas)
+
+ MAPREDUCE-1367. LocalJobRunner should support parallel mapper execution.
+ (Aaron Kimball via tomwhite)
+
+ MAPREDUCE-64. Eliminate io.sort.record.percent from MapTask configuration.
+ (cdouglas)
+
+ MAPREDUCE-1440. Replace the long user name in MapReduce with the local
+ name. (omalley)
+
+ MAPREDUCE-1470. Move delegation tokens from HDFS to Common so that
+ MapReduce can use them too. (omalley)
+
+ MAPREDUCE-1425. Reduce memory usage by archive. (mahadev via szetszwo)
+
+ MAPREDUCE-1441. Trim whitespace from directory lists pulled from the
+ configuration. (Todd Lipcon via cdouglas)
+
+ MAPREDUCE-1309. Refactor Rumen trace generator to improve code structure
+ and add extensible support for log formats. (Dick King via cdouglas)
+
+ MAPREDUCE-1503. Delegation token renewing and cancelling should provide
+ meaningful exceptions when there are failures instead of returning
+ false. (omalley)
+
+ HADOOP-6579. Upgrade commons-codec library to 1.4. (omalley)
+
+ MAPREDUCE-1423. Improve performance of CombineFileInputFormat when multiple
+ pools are configured. (Dhruba Borthakur via zshao)
+
+ MAPREDUCE-1454. Quote user supplied strings in Tracker servlets. (cdouglas)
+
+ MAPREDUCE-1408. Add customizable job submission policies to Gridmix. (Rahul
+ Singh via cdouglas)
+
+ MAPREDUCE-1527. Better warning logged when mapred.queue.names is
+ overshadowed by mapred-queues.xml. (Hong Tang via acmurthy)
+
+ MAPREDUCE-1403. Save the size and number of distributed cache artifacts in
+ the configuration. (Arun Murthy via cdouglas)
+
+ MAPREDUCE-1482. Truncate state string and diagnostic information in
+ TaskStatus. (Amar Kamat via szetszwo)
+
+ MAPREDUCE-1593. [Rumen] Improvements to random seed generation (tamas via
+ mahadev)
+
+ MAPREDUCE-1460. Oracle support in DataDrivenDBInputFormat.
+ (Aaron Kimball via tomwhite)
+
+ MAPREDUCE-1569. Pass configuration through mocked contexts in MRUnit.
+ (Chris White via cdouglas)
+
+ MAPREDUCE-1590. Move HarFileSystem from Hadoop Common to Mapreduce tools.
+ (mahadev)
+
+ MAPREDUCE-1629. Get rid of fakeBlockLocations() on HarFileSystem, since
+ it's not used (mahadev)
+
+ MAPREDUCE-1489. DataDrivenDBInputFormat should not query the database
+ when generating only one split. (Aaron Kimball via tomwhite)
+
+ MAPREDUCE-1514. Add documentation on replication, permissions, new options,
+ limitations and internals of har. (mahadev via szetszwo)
+
+ MAPREDUCE-1428. Make block size and the size of archive created files
+ configurable. (mahadev via szetszwo)
+
+ MAPREDUCE-1656. JobStory should provide queue info. (hong via mahadev)
+
+ MAPREDUCE-1466. Record number of files processed in FileInputFormat in the
+ Configuration for offline analysis. (Luke Lu and Arun Murthy via cdouglas)
+
+ MAPREDUCE-1538. TrackerDistributedCacheManager manages the
+ number of files. (Scott Chen via dhruba)
+
+ MAPREDUCE-1673. Scripts to start and stop RaidNode.
+ (Rodrigo Schmidt via dhruba)
+
+ MAPREDUCE-1659. RaidNode writes temp files on configured tmp directory and
+ add random numbers to their names to avoid conflicts
+ (Rodrigo Schmidt via dhruba)
+
+ MAPREDUCE-1221. Allow admins to control physical memory limits per-task
+ and per-node. (Scott Chen via acmurthy)
+
+ MAPREDUCE-1065. Update mapred tutorial to use the new API. (Aaron Kimball
+ via cdouglas)
+
+ MAPREDUCE-1304. Add a task counter tracking time spent in GC. (Aaron
+ Kimball via cdouglas)
+
+ MAPREDUCE-1570. Add grouping comparators to MRUnit. (Chris White via
+ cdouglas)
+
+ MAPREDUCE-1650. Exclude Private elements from generated MapReduce
+ Javadoc. (tomwhite)
+
+ MAPREDUCE-1625. Improve grouping of packages in Javadoc. (tomwhite)
+
+ MAPREDUCE-1417. Forrest documentation should be updated to reflect
+ the changes in MAPREDUCE-744. (Ravi Gummadi via vinodkv)
+
+ MAPREDUCE-1568. TrackerDistributedCacheManager should clean up cache
+ in a background thread. (Scott Chen via zshao)
+
+ MAPREDUCE-1749. Move configuration strings out of JobContext so that it
+ can be made public stable. (omalley)
+
+ MAPREDUCE-1623. Apply audience and stability notations to Hadoop
+ Map-Reduce. (tomwhite via acmurthy)
+
+ MAPREDUCE-1751. Change MapReduce to depend on Hadoop 'common' artifacts
+ instead of 'core'. (tomwhite)
+
+ MAPREDUCE-1535. Replace usage of FileStatus#isDir(). (Eli Collins via
+ tomwhite)
+
+ MAPREDUCE-1832. Allow file sizes less than 1MB in DFSIO benchmark. (shv)
+
+ MAPREDUCE-1404. Move Cluster-Setup and Single-Node-Setup Docs from
+ MapReduce to Common. (tomwhite)
+
+ MAPREDUCE-1697. Document the behavior of -file option and deprecate it
+ in favour of -files option in streaming. (Amareshwari Sriramadasu
+ via vinodkv)
+
+ MAPREDUCE-1033. Resolve location of scripts and configuration files after
+ project split. (tomwhite)
+
+ MAPREDUCE-1018. Document changes to the memory management and scheduling
+ model. (Hemanth Yamijala via vinodkv)
+
+ MAPREDUCE-1896. [Herriot] New property for multi user list. (Vinay Thota
+ via cos)
+
+ MAPREDUCE-1812. New properties for suspend and resume process. (Vinay
+ Thota via cos)
+
+ OPTIMIZATIONS
+
+ MAPREDUCE-270. Fix the tasktracker to optionally send an out-of-band
+ heartbeat on task-completion for better job-latency. (acmurthy)
+ Configuration changes:
+ add mapreduce.tasktracker.outofband.heartbeat
+
+ MAPREDUCE-1186. Modified code in distributed cache to set permissions
+ only on required set of localized paths.
+ (Amareshwari Sriramadasu via yhemanth)
+
+ MAPREDUCE-1501. FileInputFormat supports multi-level, recursive
+ directory listing. (Zheng Shao via dhruba)
+
+ MAPREDUCE-1556. upgrade to Avro 1.3.0. (cutting via tomwhite)
+
+ MAPREDUCE-1613. Install/deploy source jars to Maven repo
+ (Patrick Angeles via ddas)
+
+ MAPREDUCE-1610. Forrest documentation should be updated to reflect
+ the changes in MAPREDUCE-856. (Ravi Gummadi via vinodkv)
+
+ MAPREDUCE-1853. Adds caching for TaskAttemptContext in MultipleOutputs.
+ (Torsten Curdt via amareshwari)
+
+ BUG FIXES
+
+ MAPREDUCE-878. Rename fair scheduler design doc to
+ fair-scheduler-design-doc.tex and add Apache license header (matei)
+
+ HADOOP-4687. MapReduce is split from Hadoop Core. It is a subproject under
+ Hadoop (Owen O'Malley)
+
+ HADOOP-6096. Fix Eclipse project and classpath files following project
+ split. (tomwhite)
+
+ MAPREDUCE-419. Reconcile mapred.userlog.limit.kb defaults in configuration
+ and code. (Philip Zeyliger via cdouglas)
+
+ MAPREDUCE-2. Fixes a bug in KeyFieldBasedPartitioner in handling empty
+ keys. (Amar Kamat via sharad)
+
+ MAPREDUCE-130. Delete the jobconf copy from the log directory of the
+ JobTracker when the job is retired. (Amar Kamat via sharad)
+
+ MAPREDUCE-657. Fix hardcoded filesystem problem in CompletedJobStatusStore.
+ (Amar Kamat via sharad)
+
+ MAPREDUCE-179. Update progress in new RecordReaders. (cdouglas)
+
+ MAPREDUCE-658. Replace NPE in distcp with a meaningful error message when
+ the source path does not exist. (Ravi Gummadi via cdouglas)
+
+ MAPREDUCE-671. Update ignore list to include untracked, generated
+ build artifacts and config files. (cdouglas)
+
+ MAPREDUCE-433. Use more reliable counters in TestReduceFetch. (cdouglas)
+
+ MAPREDUCE-124. Fix a bug in failure handling of abort task of
+ OutputCommiter. (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-694. Fix to add jsp-api jars to capacity-scheduler classpath.
+ (Giridharan Kesavan)
+
+ MAPREDUCE-702. Fix eclipse-plugin jar target (Giridharan Kesavan)
+
+ MAPREDUCE-522. Replace TestQueueCapacities with simpler test case to
+ test integration between capacity scheduler and MR framework.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-683. Fixes an initialization problem in the JobHistory.
+ The initialization of JobHistoryFilesManager is now done in the
+ JobHistory.init call. (Amar Kamat via ddas)
+
+ MAPREDUCE-708. Fixes a bug to allow updating the reason for
+ blacklisting a node on the JobTracker UI.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-709. Fixes message displayed for a blacklisted node where
+ the reason for blacklisting is due to the health check script
+ timing out. (Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-676. Existing diagnostic rules fail for MAP ONLY jobs.
+ (Suhas Gogate via tomwhite)
+
+ MAPREDUCE-722. Fixes a bug with tasktracker reservations for
+ high memory jobs in capacity scheduler.
+ (Vinod Kumar Vavilapalli via yhemanth)
+
+ HADOOP-6090. Updates gridmix script to use new mapreduce api output
+ format. (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-732. Removed spurious log statements in the node
+ blacklisting logic. (Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-734. Fix a ConcurrentModificationException in unreserving
+ unused reservations for a job when it completes.
+ (Arun Murthy and Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-733. Fix a RuntimeException while unreserving trackers
+ that are blacklisted for a job.
+ (Arun Murthy and Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-677. Fix timeout in TestNodeRefresh. (Amar Kamat via
+ sharad)
+
+ MAPREDUCE-153. Fix timeout in TestJobInProgressListener. (Amar
+ Kamat via sharad)
+
+ MAPREDUCE-742. Fix output messages and java comments in the Pi related
+ examples. (szetszwo)
+
+ MAPREDUCE-565. Fix partitioner to work with new API. (Owen O'Malley via
+ cdouglas)
+
+ MAPREDUCE-680. Fix so MRUnit can handle reuse of Writable objects.
+ (Aaron Kimball via johan)
+
+ MAPREDUCE-18. Puts some checks for cross checking whether a reduce
+ task gets the correct shuffle data. (Ravi Gummadi via ddas)
+
+ MAPREDUCE-771. Fix scheduling of setup and cleanup tasks to use
+ free slots instead of tasks for scheduling. (yhemanth)
+
+ MAPREDUCE-717. Fixes some corner case issues in speculative
+ execution heuristics. (Devaraj Das)
+
+ MAPREDUCE-716. Make DBInputFormat work with Oracle. (Aaron Kimball
+ via tomwhite)
+
+ MAPREDUCE-735. Fixes a problem in the KeyFieldHelper to do with
+ the end index for some inputs (Amar Kamat via ddas)
+
+ MAPREDUCE-682. Removes reservations on tasktrackers which are
+ blacklisted. (Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-743. Fixes a problem to do with progress reporting
+ in the map phase. (Ravi Gummadi via ddas)
+
+ MAPREDUCE-765. Eliminate the deprecated warnings introduced by H-5438.
+ (He Yongqiang via szetszwo)
+
+ MAPREDUCE-383. Fix a bug in Pipes combiner due to bytes count not
+ getting reset after the spill. (Christian Kunz via sharad)
+
+ MAPREDUCE-809. Fix job-summary logs to correctly record status of FAILED
+ and KILLED jobs. (acmurthy)
+
+ MAPREDUCE-792. Fix unchecked warnings in DBInputFormat. (Aaron Kimball
+ via szetszwo)
+
+ MAPREDUCE-760. Fix a timing issue in TestNodeRefresh. (Amar Kamat via
+ sharad)
+
+ MAPREDUCE-40. Keep memory management backwards compatible for job
+ configuration parameters and limits. (Rahul Kumar Singh via yhemanth)
+
+ MAPREDUCE-587. Fixes a OOM issue in TestStreamingExitStatus.
+ (Amar Kamat via ddas)
+
+ MAPREDUCE-408. Fixes an assertion problem in TestKillSubProcesses
+ (Ravi Gummadi via ddas)
+
+ MAPREDUCE-659. Fix gridmix2 compilation. (Giridharan Kesavan)
+
+ MAPREDUCE-796. Fixes a ClassCastException in an exception log in
+ MultiThreadedMapRunner. (Amar Kamat via ddas)
+
+ MAPREDUCE-808. Fixes a serialization problem in TypedBytes.
+ (Klaas Bosteels via ddas)
+
+ MAPREDUCE-845. Fix a findbugs heap size problem in build.xml and add
+ a new property findbugs.heap.size. (Lee Tucker via szetszwo)
+
+ MAPREDUCE-838. Fixes a problem in the way commit of task outputs
+ happens. The bug was that even if commit failed, the task would
+ be declared as successful. (Amareshwari Sriramadasu via ddas)
+
+ MAPREDUCE-813. Updates Streaming and M/R tutorial documents.
+ (Corinne Chandel via ddas)
+
+ MAPREDUCE-805. Fixes some deadlocks in the JobTracker due to the fact
+ the JobTracker lock hierarchy wasn't maintained in some JobInProgress
+ method calls. (Amar Kamat via ddas)
+
+ MAPREDUCE-799. Fixes so all of the MRUnit self-tests run.
+ (Aaron Kimball via johan)
+
+ MAPREDUCE-848. Fixes a problem to do with TestCapacityScheduler
+ failing (Amar Kamat via ddas)
+
+ MAPREDUCE-840. DBInputFormat leaves open transaction.
+ (Aaron Kimball via tomwhite)
+
+ MAPREDUCE-859. Adds Avro and its dependencies required by Hadoop
+ common. (Ravi Gummadi via sharad)
+
+ MAPREDUCE-867. Fix ivy conf to look for avro jar from maven repo.
+ (Giridharan Kesavan)
+
+ MAPREDUCE-877. Added avro as a dependency to contrib ivy settings.
+ (Tsz Wo (Nicholas) Sze via yhemanth)
+
+ MAPREDUCE-852. In build.xml, remove the Main-Class, which is incorrectly
+ set in tools, and rename the target "tools-jar" to "tools". (szetszwo)
+
+ MAPREDUCE-773. Sends progress reports for compressed gzip inputs in maps.
+ Fixes a native direct buffer leak in LineRecordReader classes.
+ (Hong Tang and ddas)
+
+ MAPREDUCE-832. Reduce number of warning messages printed when
+ deprecated memory variables are used. (Rahul Kumar Singh via yhemanth)
+
+ MAPREDUCE-745. Fixes a testcase problem to do with generation of JobTracker
+ IDs. (Amar Kamat via ddas)
+
+ MAPREDUCE-834. Enables memory management on tasktrackers when old
+ memory management parameters are used in configuration.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-818. Fixes Counters#getGroup API. (Amareshwari Sriramadasu
+ via sharad)
+
+ MAPREDUCE-807. Handles the AccessControlException during the deletion of
+ mapred.system.dir in the JobTracker. The JobTracker will bail out if it
+ encounters such an exception. (Amar Kamat via ddas)
+
+ MAPREDUCE-430. Fix a bug related to task getting stuck in case of
+ OOM error. (Amar Kamat via ddas)
+
+ MAPREDUCE-871. Fix ownership of Job/Task local files to have correct
+ group ownership according to the egid of the tasktracker.
+ (Vinod Kumar Vavilapalli via yhemanth)
+
+ MAPREDUCE-911. Fix a bug in TestTaskFail related to speculative
+ execution. (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-687. Fix an assertion in TestMiniMRMapRedDebugScript.
+ (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-924. Fixes the TestPipes testcase to use Tool.
+ (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-903. Add Avro jar to eclipse classpath.
+ (Philip Zeyliger via tomwhite)
+
+ MAPREDUCE-943. Removes a testcase in TestNodeRefresh that doesn't make
+ sense in the new Job recovery model. (Amar Kamat via ddas)
+
+ MAPREDUCE-764. TypedBytesInput's readRaw() does not preserve custom type
+ codes. (Klaas Bosteels via tomwhite)
+
+ HADOOP-6243. Fixes a NullPointerException in handling deprecated keys.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-968. NPE in distcp encountered when placing _logs directory on
+ S3FileSystem. (Aaron Kimball via tomwhite)
+
+ MAPREDUCE-826. harchive doesn't use ToolRunner / harchive returns 0 even
+ if the job fails with exception (koji Noguchi via mahadev)
+
+ MAPREDUCE-839. unit test TestMiniMRChildTask fails on mac os-x (hong tang
+ via mahadev)
+
+ MAPREDUCE-112. Add counters for reduce input, output records to the new API.
+ (Jothi Padmanabhan via cdouglas)
+
+ MAPREDUCE-648. Fix two distcp bugs: (1) it should not launch a job if all
+ src paths are directories, and (2) it does not skip copying when updating
+ a single file. (Ravi Gummadi via szetszwo)
+
+ MAPREDUCE-946. Fix a regression in LineRecordReader where the
+ maxBytesToConsume parameter is not set correctly. (cdouglas)
+
+ MAPREDUCE-977. Missing jackson jars from Eclipse template. (tomwhite)
+
+ MAPREDUCE-988. Fix a packaging issue in the contrib modules. (Hong Tang via
+ cdouglas)
+
+ MAPREDUCE-971. distcp does not always remove distcp.tmp.dir. (Aaron Kimball
+ via tomwhite)
+
+ MAPREDUCE-995. Fix a bug in JobHistory where tasks completing after the job
+ is closed cause a NPE. (Jothi Padmanabhan via cdouglas)
+
+ MAPREDUCE-953. Fix QueueManager to dump queue configuration in JSON format.
+ (V.V. Chaitanya Krishna via yhemanth)
+
+ MAPREDUCE-645. Prevent distcp from running a job when the destination is a
+ file, but the source is not. (Ravi Gummadi via cdouglas)
+
+ MAPREDUCE-1002. Flushed writer in JobQueueClient so queue information is
+ printed correctly. (V.V. Chaitanya Krishna via yhemanth)
+
+ MAPREDUCE-1003. Fix compilation problem in eclipse plugin when
+ eclipse.home is set. (Ravi Gummadi via yhemanth)
+
+ MAPREDUCE-941. Vaidya script fails on Solaris. (Chad Metcalf
+ via tomwhite)
+
+ MAPREDUCE-912. Add and standardize Apache license headers. (Chad Metcalf
+ via cdouglas)
+
+ MAPREDUCE-1022. Fix compilation of vertica testcases. (Vinod Kumar
+ Vavilapalli via acmurthy)
+
+ MAPREDUCE-1000. Handle corrupt history files in JobHistory.initDone().
+ (Jothi Padmanabhan via sharad)
+
+ MAPREDUCE-1028. Fixed number of slots occupied by cleanup tasks to one
+ irrespective of slot size for the job.
+ (Ravi Gummadi via yhemanth)
+
+ MAPREDUCE-964. Fixed start and finish times of TaskStatus to be
+ consistent, thereby fixing inconsistencies in metering tasks.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-1076. Deprecate ClusterStatus and add javadoc in ClusterMetrics.
+ (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-979. Fixed JobConf APIs related to memory parameters to return
+ values of new configuration variables when deprecated variables are
+ disabled. (Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-1030. Modified scheduling algorithm to return a map and reduce
+ task per heartbeat in the capacity scheduler.
+ (Rahul Kumar Singh via yhemanth)
+
+ MAPREDUCE-1071. Use DataInputStream rather than FSDataInputStream in the
+ JobHistory EventReader. (Hong Tang via cdouglas)
+
+ MAPREDUCE-986. Fix Rumen to work with truncated task lines. (Dick King via
+ cdouglas)
+
+ MAPREDUCE-1029. Fix failing TestCopyFiles by restoring the unzipping of
+ HDFS webapps from the hdfs jar. (Aaron Kimball and Jothi Padmanabhan via
+ cdouglas)
+
+ MAPREDUCE-769. Make findbugs and javac warnings to zero.
+ (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-1104. Initialize RecoveryManager in JobTracker cstr called by
+ Mumak. (Hong Tang via cdouglas)
+
+ MAPREDUCE-1061. Add unit test validating byte specifications for gridmix
+ jobs. (cdouglas)
+
+ MAPREDUCE-1077. Fix Rumen so that truncated tasks do not mark the job as
+ successful. (Dick King via cdouglas)
+
+ MAPREDUCE-1041. Make TaskInProgress::taskStatuses map package-private.
+ (Jothi Padmanabhan via cdouglas)
+
+ MAPREDUCE-1070. Prevent a deadlock in the fair scheduler servlet.
+ (Todd Lipcon via cdouglas)
+
+ MAPREDUCE-1086. Setup Hadoop logging environment for tasks to point to
+ task related parameters. (Ravi Gummadi via yhemanth)
+
+ MAPREDUCE-1105. Remove max limit configuration in capacity scheduler in
+ favor of max capacity percentage thus allowing the limit to go over
+ queue capacity. (Rahul Kumar Singh via yhemanth)
+
+ MAPREDUCE-1016. Make the job history log format JSON. (cutting)
+
+ MAPREDUCE-1038. Weave Mumak aspects only if related files have changed.
+ (Aaron Kimball via cdouglas)
+
+ MAPREDUCE-1163. Remove unused, hard-coded paths from libhdfs. (Allen
+ Wittenauer via cdouglas)
+
+ MAPREDUCE-962. Fix a NullPointerException while killing task process
+ trees. (Ravi Gummadi via yhemanth)
+
+ MAPREDUCE-1177. Correct setup/cleanup inversion in
+ JobTracker::getTaskReports. (Vinod Kumar Vavilapalli via cdouglas)
+
+ MAPREDUCE-1178. Fix ClassCastException in MultipleInputs by adding
+ a DelegatingRecordReader. (Amareshwari Sriramadasu and Jay Booth
+ via sharad)
+
+ MAPREDUCE-1068. Fix streaming job to show proper message if file is
+ is not present. (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-1147. Add map output counters to new API. (Amar Kamat via
+ cdouglas)
+
+ MAPREDUCE-915. The debug scripts are run as the job user. (ddas)
+
+ MAPREDUCE-1007. Fix NPE in CapacityTaskScheduler.getJobs().
+ (V.V.Chaitanya Krishna via sharad)
+
+ MAPREDUCE-28. Refactor TestQueueManager and fix default ACLs.
+ (V.V.Chaitanya Krishna and Rahul K Singh via sharad)
+
+ MAPREDUCE-1182. Fix overflow in reduce causing allocations to exceed the
+ configured threshold. (cdouglas)
+
+ MAPREDUCE-1239. Fix contrib components build dependencies.
+ (Giridharan Kesavan and omalley)
+
+ MAPREDUCE-787. Fix JobSubmitter to honor user given symlink path.
+ (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-1249. Update config default value for socket read timeout to
+ match code default. (Amareshwari Sriramadasu via cdouglas)
+
+ MAPREDUCE-1161. Remove ineffective synchronization in NotificationTestCase.
+ (Owen O'Malley via cdouglas)
+
+ MAPREDUCE-1244. Fix eclipse-plugin's build dependencies. (gkesavan)
+
+ MAPREDUCE-1075. Fix JobTracker to not throw an NPE for a non-existent
+ queue. (V.V.Chaitanya Krishna via yhemanth)
+
+ MAPREDUCE-754. Fix NPE in expiry thread when a TT is lost. (Amar Kamat
+ via sharad)
+
+ MAPREDUCE-1074. Document Reducer mark/reset functionality. (Jothi
+ Padmanabhan via cdouglas)
+
+ MAPREDUCE-1267. Fix typo in mapred-default.xml. (Todd Lipcon via cdouglas)
+
+ MAPREDUCE-952. Remove inadvertently reintroduced Task.Counter enum. (Jothi
+ Padmanabhan via cdouglas)
+
+ MAPREDUCE-1230. Fix handling of null records in VerticaInputFormat. (Omer
+ Trajman via cdouglas)
+
+ MAPREDUCE-1171. Allow shuffle retries and read-error reporting to be
+ configurable. (Amareshwari Sriramadasu via acmurthy)
+
+ MAPREDUCE-879. Fix broken unit test TestTaskTrackerLocalization on MacOS.
+ (Sreekanth Ramakrishnan via yhemanth)
+
+ MAPREDUCE-1124. Fix imprecise byte counts in Gridmix. (cdouglas)
+
+ MAPREDUCE-1222. Add an option to exclude numeric IP addresses in topologies
+ processed by Mumak. (Hong Tang via cdouglas)
+
+ MAPREDUCE-1284. Fix fts_open() call in task-controller that was failing
+ LinuxTaskController unit tests. (Ravi Gummadi via yhemanth)
+
+ MAPREDUCE-1143. Fix running task counters to be updated correctly
+ when speculative attempts are running for a TIP.
+ (Rahul Kumar Singh via yhemanth)
+
+ MAPREDUCE-1241. Use a default queue configuration in JobTracker when
+ mapred-queues.xml is unavailable. (Todd Lipcon via cdouglas)
+
+ MAPREDUCE-1301. Fix set up of permission checking script used in
+ localization tests. (Amareshwari Sriramadasu via yhemanth)
+
+ MAPREDUCE-1286. Remove quoting from client opts in TaskRunner. (Yuri
+ Pradkin via cdouglas)
+
+ MAPREDUCE-1059. Use distcp.bytes.per.map when adding sync markers in
+ distcp. (Aaron Kimball via cdouglas)
+
+ MAPREDUCE-1009. Update forrest documentation describing hierarchical
+ queues. (Vinod Kumar Vavilapalli via yhemanth)
+
+ MAPREDUCE-1342. Fixed deadlock in global blacklisting of tasktrackers.
+ (Amareshwari Sriramadasu via acmurthy)
+
+ MAPREDUCE-1316. Fixes a memory leak of TaskInProgress instances in
+ the jobtracker. (Amar Kamat via yhemanth)
+
+ MAPREDUCE-1359. TypedBytes TestIO doesn't mkdir its test dir first.
+ (Anatoli Fomenko via cos)
+
+ MAPREDUCE-1314. Correct errant mapreduce.x.mapreduce.x replacements from
+ bulk change. (Amareshwari Sriramadasu via cdouglas)
+
+ MAPREDUCE-1365. Restore accidentally renamed test in
+ TestTaskTrackerBloacklisting. (Amareshwari Sriramadasu via cdouglas)
+
+ MAPREDUCE-1406. Fix spelling of JobContext.MAP_COMBINE_MIN_SPILLS.
+ (cdouglas)
+
+ MAPREDUCE-1369. JUnit tests should never depend on anything in conf
+ (Anatoli Fomenko via cos)
+
+ MAPREDUCE-1412. Fix timer granularity issue causing failures in
+ TestTaskTrackerBlacklisting. (cdouglas)
+
+ MAPREDUCE-1448. Respect --config option in Mumak script. (Hong Tang via
+ cdouglas)
+
+ MAPREDUCE-1251. c++ utils doesn't compile. (Eli Collins via tomwhite)
+
+ MAPREDUCE-1522. FileInputFormat may use the default FileSystem for the
+ input path. (Tsz Wo (Nicholas), SZE via cdouglas)
+
+ MAPREDUCE-1407. Update javadoc in mapreduce.{Mapper,Reducer} to match
+ actual usage. (Benoit Sigoure via cdouglas)
+
+ MAPREDUCE-1258. Fix fair scheduler event log not logging job info.
+ (matei)
+
+ MAPREDUCE-1089. Fix NPE in fair scheduler preemption when tasks are
+ scheduled but not running. (Todd Lipcon via matei)
+
+ MAPREDUCE-1014. Fix the libraries for common and hdfs. (omalley)
+
+ MAPREDUCE-1111. JT Jetty UI not working if we run mumak.sh
+ off packaged distribution directory. (hong tang via mahadev)
+
+ MAPREDUCE-1133. Eclipse .classpath template has outdated jar files and is
+ missing some new ones. (cos)
+
+ MAPREDUCE-1098. Fixed the distributed-cache to not do i/o while holding a
+ global lock. (Amareshwari Sriramadasu via acmurthy)
+
+ MAPREDUCE-1158. Fix JT running maps and running reduces metrics.
+ (sharad)
+
+ MAPREDUCE-1160. Reduce verbosity of log lines in some Map/Reduce classes
+ to avoid filling up jobtracker logs on a busy cluster.
+ (Ravi Gummadi and Hong Tang via yhemanth)
+
+ MAPREDUCE-1153. Fix tasktracker metrics when trackers are decommissioned.
+ (sharad)
+
+ MAPREDUCE-1128. Fix MRUnit to prohibit iterating over values twice. (Aaron
+ Kimball via cdouglas)
+
+ MAPREDUCE-665. Move libhdfs to HDFS subproject. (Eli Collins via dhruba)
+
+ MAPREDUCE-1196. Fix FileOutputCommitter to use the deprecated cleanupJob
+ api correctly. (acmurthy)
+
+ MAPREDUCE-1244. Fix eclipse-plugin's build dependencies. (gkesavan)
+
+ MAPREDUCE-1140. Fix DistributedCache to not decrement reference counts for
+ unreferenced files in error conditions.
+ (Amareshwari Sriramadasu via yhemanth)
+
+ MAPREDUCE-1245. Fix TestFairScheduler failures by instantiating lightweight
+ Jobtracker. (sharad)
+
+ MAPREDUCE-1260. Update Eclipse configuration to match changes to Ivy
+ configuration. (Edwin Chan via cos)
+
+ MAPREDUCE-1152. Distinguish between failed and killed tasks in
+ JobTrackerInstrumentation. (Sharad Agarwal via cdouglas)
+
+ MAPREDUCE-1285. In DistCp.deleteNonexisting(..), get class from the
+ parameter instead of using FileStatus.class. (Peter Romianowski via
+ szetszwo)
+
+ MAPREDUCE-1294. Build fails to pull latest hadoop-core-* artifacts (cos)
+
+ MAPREDUCE-1213. TaskTrackers restart is faster because it deletes
+ distributed cache directory asynchronously. (Zheng Shao via dhruba)
+
+ MAPREDUCE-1265. The task attempt error log prints the name of the
+ tasktracker machine. (Scott Chen via dhruba)
+
+ MAPREDUCE-1201. ProcfsBasedProcessTree collects CPU usage information.
+ (Scott Chen via dhruba)
+
+ MAPREDUCE-1326. fi tests don't use fi-site.xml (cos)
+
+ MAPREDUCE-1165. Replace non-portable function name with C99 equivalent.
+ (Allen Wittenauer via cdouglas)
+
+ MAPREDUCE-1331. Fixes a typo in a testcase (Devaraj Das)
+
+ MAPREDUCE-1293. AutoInputFormat doesn't work with non-default FileSystems.
+ (Andrew Hitchcock via tomwhite)
+
+ MAPREDUCE-1131. Using profilers other than hprof can cause JobClient to
+ report job failure. (Aaron Kimball via tomwhite)
+
+ MAPREDUCE-1155. Streaming tests swallow exceptions.
+ (Todd Lipcon via tomwhite)
+
+ MAPREDUCE-1212. Mapreduce contrib project ivy dependencies are not included
+ in binary target. (Aaron Kimball via tomwhite)
+
+ MAPREDUCE-1388. Move the HDFS RAID package from HDFS to MAPREDUCE.
+ (Eli Collins via dhruba)
+
+ MAPREDUCE-1322. Defines default value for staging directory to be user
+ based fixing a failing streaming test.
+ (Devaraj Das and Amareshwari Sriramadasu via yhemanth)
+
+ MAPREDUCE-899. Modified LinuxTaskController to check that task-controller
+ has right permissions and ownership before performing any actions.
+ (Amareshwari Sriramadasu via yhemanth)
+
+ MAPREDUCE-1443. DBInputFormat can leak connections.
+ (Aaron Kimball via tomwhite)
+
+ MAPREDUCE-1457. Fixes JobTracker to get the FileSystem object within
+ getStagingAreaDir within a privileged block. Fixes Child.java to use the
+ appropriate UGIs while getting the TaskUmbilicalProtocol proxy and
+ while executing the task. (Jakob Homan via ddas)
+
+ MAPREDUCE-1399. The archive command shows a null error message (nicholas
+ via mahadev)
+
+ MAPREDUCE-1305. Improve efficiency of distcp -delete. (Peter Romianowski
+ via cdouglas)
+
+ MAPREDUCE-1474. Update forrest documentation for Hadoop Archives. (Mahadev
+ Konar via cdouglas)
+
+ MAPREDUCE-1400. Use tr rather than sed to effect literal substitution in
+ the build script. (Allen Wittenauer via cdouglas)
+
+ MAPREDUCE-1358. Avoid false positives in OutputLogFilter. (Todd Lipcon via
+ cdouglas)
+
+ MAPREDUCE-1490. Fix a NullPointerException that could occur during
+ instantiation and initialization of the DistributedRaidFileSystem.
+ (Rodrigo Schmidt via dhruba)
+
+ MAPREDUCE-1476. Fix the M/R framework to not call commit for special
+ tasks like job setup/cleanup and task cleanup.
+ (Amareshwari Sriramadasu via yhemanth)
+
+ MAPREDUCE-1398. Fix TaskLauncher to stop waiting for slots on a TIP that
+ is killed / failed.
+ (Amareshwari Sriramadasu via yhemanth)
+
+ MAPREDUCE-1491. The parity files created by the RAID are combined
+ using Hadoop Archive Files (HAR). (Rodrigo Schmidt via dhruba)
+
+ MAPREDUCE-1378. URL encode link in jobhistory.jsp to avoid errors caused by
+ unescaped characters. (E. Sammer via cdouglas)
+
+ MAPREDUCE-1519. RaidNode fails to create new parity file
+ if an older version already exists. (Rodrigo Schmidt via dhruba)
+
+ MAPREDUCE-1537. Fixes a compilation problem in a testcase after commit
+ HDFS-984. (Jitendra Nath Pandey via ddas)
+
+ MAPREDUCE-1537. The patch makes the job client call the getDelegationToken
+ only when security is enabled. (Jitendra Nath Pandey via ddas)
+
+ MAPREDUCE-1510. RAID should regenerate parity files if they get deleted.
+ (Rodrigo Schmidt via dhruba)
+
+ MAPREDUCE-1421. Fix the LinuxTaskController tests failing on trunk after
+ the commit of MAPREDUCE-1385. (Amareshwari Sriramadasu via vinodkv)
+
+ MAPREDUCE-1520. Fix TestMiniMRLocalFS failure caused by regression in
+ getting user working dir. (Amareshwari Sriramadasu via cdouglas)
+
+ MAPREDUCE-1512. RAID uses HarFileSystem directly instead of
+ FileSystem.get (Rodrigo Schmidt via dhruba)
+
+ MAPREDUCE-1435. Fix symlink handling in task work directory when
+ cleaning up, essentially to avoid following links.
+ (Ravi Gummadi via yhemanth)
+
+ MAPREDUCE-1518. RaidNode does not run the deletion check on the
+ directory that stores the parity files. (Rodrigo Schmidt via dhruba)
+
+ MAPREDUCE-1573. TestStreamingAsDifferentUser fails if run as tt_user.
+ (Ravi Gummadi via vinodkv)
+
+ MAPREDUCE-927. Cleanup of task-logs should happen in TaskTracker instead
+ of the Child. (Amareshwari Sriramadasu via vinodkv)
+
+ MAPREDUCE-1578. Decouple HadoopArchives vesrion from HarFileSystem version.
+ (Rodrigo Schmidt via szetszwo)
+
+ MAPREDUCE-1422. Fix cleanup of localized job directory to work if files
+ with non-deletable permissions are created within it.
+ (Amar Kamat via yhemanth)
+
+ MAPREDUCE-1306. Randomize the arrival of heartbeat responses in Mumak.
+ (Tamas Sarlos via cdouglas)
+
+ MAPREDUCE-1579. archive: check and possibly replace the space charater
+ in source paths. (szetszwo)
+
+ MAPREDUCE-1536. DataDrivenDBInputFormat does not split date columns correctly.
+ (Aaron Kimball via enis)
+
+ MAPREDUCE-890. After HADOOP-4491, the user who started mapred system is
+ not able to run job. (Ravi Gummadi via vinodkv)
+
+ MAPREDUCE-1615. Fix compilation of TestSubmitJob. (cdouglas)
+
+ MAPREDUCE-1508. Protect against NPE in TestMultipleLevelCaching. (Aaron
+ Kimball via cdouglas)
+
+ MAPREDUCE-1497. Suppress spurious findbugs warning about IndexCache
+ synchronization. (Amareshwari Sriramadasu via cdouglas)
+
+ MAPREDUCE-1420. Fix TestTTResourceReporting failure. (Scott Chen via
+ cdouglas)
+
+ MAPREDUCE-1480. Correctly initialize child RecordReaders in
+ CombineFileInputFormat. (Aaron Kimball via cdouglas)
+
+ MAPREDUCE-1348. Fix block forensics packaging. (Tom White via cdouglas)
+
+ MAPREDUCE-1628. HarFileSystem shows incorrect replication numbers and
+ permissions. (szetszwo via mahadev)
+
+ MAPREDUCE-1602. Fix the error message for the case that src does not
+ exist. (szetszwo)
+
+ MAPREDUCE-1585. Create Hadoop Archives version 2 with filenames
+ URL-encoded (rodrigo via mahadev)
+
+ MAPREDUCE-1523. Sometimes rumen trace generator fails to extract the job
+ finish time. (dick king via mahadev)
+
+ MAPREDUCE-1635. ResourceEstimator does not work after MAPREDUCE-842.
+ (Amareshwari Sriramadasu via vinodkv)
+
+ MAPREDUCE-889. binary communication formats added to Streaming by
+ HADOOP-1722 should be documented. (Klaas Bosteels via tomwhite)
+
+ MAPREDUCE-1031. ant tar target doens't seem to compile tests in contrib
+ projects. (Aaron Kimball via tomwhite)
+
+ MAPREDUCE-1692. Removed unused testcase TestStreamedMerge.
+ (Sreekanth Ramakrishnan and Amareshwari Sriramadasu via yhemanth)
+
+ MAPREDUCE-1062. Fix ReliabilityTest to work with retired jobs. (Sreekanth
+ Ramakrishnan via cdouglas)
+
+ MAPREDUCE-1409. IOExceptions thrown from FIleOutputCommitter::abortTask
+ should cause the task to fail. (Amareshwari Sriramadasu via cdouglas)
+
+ MAPREDUCE-1695. Include capacity scheduler in findbugs and javadoc-dev
+ targets and also fix existing warnings. (Hong Tang via yhemanth)
+
+ MAPREDUCE-1494. Ensure TestJobDirCleanup verifies the correct paths.
+ (Amareshwari Sriramadasu via cdouglas)
+
+ MAPREDUCE-1622. Include missing slf4j dependencies. (cdouglas)
+
+ MAPREDUCE-1515. Accept java5.home from build.properties, not only from the
+ command line when building forrest docs. (Al Thompson via cdouglas)
+
+ MAPREDUCE-1618. Add missing javadoc to JobStatus::*JobAcls. (Amareshwari
+ Sriramadasu via cdouglas)
+
+ MAPREDUCE-1219. Remove job level metrics from jobtracker metrics to ease
+ undue load on jobtracker. (Sreekanth Ramakrishnan via sharad)
+
+ MAPREDUCE-1604. Add Forrest documentation for Job ACLs.
+ (Amareshwari Sriramadasu via yhemanth)
+
+ MAPREDUCE-1705. Archiving and Purging of HDFS parity files should
+ handle globbed policies accurately. (Rodrigo Schmidt via dhruba)
+
+ MAPREDUCE-1612. job conf file is not accessible from job history web page.
+ (Ravi Gummadi and Sreekanth Ramakrishnan via vinodkv)
+
+ MAPREDUCE-1397. NullPointerException observed during task failures.
+ (Amareshwari Sriramadasu via vinodkv)
+
+ MAPREDUCE-1728. Oracle timezone strings do not match Java.
+ (Aaron Kimball via tomwhite)
+
+ MAPREDUCE-1609. TaskTracker.localizeJob should not set permissions on
+ job log directory recursively. (Amareshwari Sriramadasu via vinodkv)
+
+ MAPREDUCE-1657. After task logs directory is deleted, tasklog servlet
+ displays wrong error message about job ACLs. (Ravi Gummadi via vinodkv)
+
+ MAPREDUCE-1727. TestJobACLs fails after HADOOP-6686. (Ravi Gummadi via vinodkv)
+
+ MAPREDUCE-1611. Refresh nodes and refresh queues doesnt work with service
+ authorization enabled. (Amar Kamat via vinodkv)
+
+ MAPREDUCE-1276. Correct flaws in the shuffle related to connection setup
+ and failure attribution. (Amareshwari Sriramadasu via cdouglas)
+
+ MAPREDUCE-1372. ConcurrentModificationException in JobInProgress.
+ (Dick King and Amareshwari Sriramadasu via tomwhite)
+
+ MAPREDUCE-118. Fix Job.getJobID(). (Amareshwari Sriramadasu via sharad)
+
+ MAPREDUCE-913. TaskRunner crashes with NPE resulting in held up slots,
+ UNINITIALIZED tasks and hung TaskTracker. (Amareshwari Sriramadasu and
+ Sreekanth Ramakrishnan via vinodkv)
+
+ MAPREDUCE-1725. Fix MapReduce API incompatibilities between 0.20 and 0.21.
+ (tomwhite)
+
+ MAPREDUCE-1606. TestJobACLs may timeout as there are no slots for launching
+ JOB_CLEANUP task. (Ravi Gummadi via vinodkv)
+
+ MAPREDUCE-1765. Correct streaming documentation for StreamXmlRecordReader.
+ (Corinne Chandel via amareshwari)
+
+ MAPREDUCE-1880. Fix BigDecimal.divide(..) in the pi example. (szetszwo)
+
+ MAPREDUCE-1885. Revert FileSystem create method that takes CreateFlags
+ (MapReduce part of HADOOP-6826). (Ravi Gummadi via tomwhite)
+
+ MAPREDUCE-1870. Harmonize MapReduce JAR library versions with Common and
+ HDFS. (tomwhite)
+
+ MAPREDUCE-1791. Remote cluster control functionality needs JavaDocs
+ improvement (Konstantin Boudnik)
+
+ MAPREDUCE-1942. 'compile-fault-inject' should never be called directly.
+ (Konstantin Boudnik)
+
+ MAPREDUCE-1876. Fixes TaskAttemptStartedEvent to correctly log event type
+ for all task types. (Amar Kamat via amareshwari)
+
+ MAPREDUCE-1926. MapReduce distribution is missing build-utils.xml.
+ (tomwhite)
+
+ MAPREDUCE-2012. Some contrib tests fail in branch 0.21 and trunk.
+ (Amareshwari Sriramadasu via tomwhite)
+
+ MAPREDUCE-1980. Fixes TaskAttemptUnsuccessfulCompletionEvent and
+ TaskAttemptFinishedEvent to correctly log event type for all task types.
+ (Amar Kamat via amareshwari)
+
+ MAPREDUCE-1856. Extract a subset of tests for smoke (DOA) validation (cos)
+