You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by gk...@apache.org on 2011/12/14 20:11:15 UTC
svn commit: r1214410 [2/3] - in /hadoop/common/branches/branch-1.0:
CHANGES.txt ivy/hadoop-core-pom-template.xml
Modified: hadoop/common/branches/branch-1.0/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1.0/CHANGES.txt?rev=1214410&r1=1214409&r2=1214410&view=diff
==============================================================================
--- hadoop/common/branches/branch-1.0/CHANGES.txt (original)
+++ hadoop/common/branches/branch-1.0/CHANGES.txt Wed Dec 14 19:11:15 2011
@@ -4,7 +4,7 @@ Release 1.0.0 - 2011.11.27
NEW FEATURES
- HDFS-2316. [umbrella] WebHDFS: a complete FileSystem implementation for
+ HDFS-2316. [umbrella] WebHDFS: a complete FileSystem implementation for
accessing HDFS over HTTP (szetszwo)
HDFS-2539. Support doAs and GETHOMEDIRECTORY in WebHDFS.
@@ -17,7 +17,7 @@ Release 1.0.0 - 2011.11.27
HDFS-2501. Add version prefix and root methods to WebHDFS. (szetszwo)
- HADOOP-7728. Enable task memory management to be configurable in hadoop
+ HADOOP-7728. Enable task memory management to be configurable in hadoop
config setup script. (ramya)
HDFS-2450. Filesystem supports path with both short names and FQDN.
@@ -26,7 +26,7 @@ Release 1.0.0 - 2011.11.27
HDFS-617. Support for non-recursive create() in HDFS.
(Kan Zhang via jitendra)
- HADOOP-6840. Support non-recursive create() in FileSystem &
+ HADOOP-6840. Support non-recursive create() in FileSystem &
SequenceFile.Writer. (Nicolas Spiegelberg via jitendra)
HADOOP-6886. LocalFileSystem Needs createNonRecursive API.
@@ -39,7 +39,7 @@ Release 1.0.0 - 2011.11.27
if the override value is same as the final parameter value.
(Ravi Prakash via szetszwo)
- HADOOP-7816. Allow HADOOP_HOME deprecated warning suppression based
+ HADOOP-7816. Allow HADOOP_HOME deprecated warning suppression based
on config specified in hadoop-env.sh (Dave Thompson via suresh)
MAPREDUCE-3169. Create a new MiniMRCluster equivalent which only provides
@@ -47,10 +47,10 @@ Release 1.0.0 - 2011.11.27
HDFS-2552. Add Forrest doc for WebHDFS REST API. (szetszwo)
- HDFS-2246. Shortcut a local client reads to a Datanodes files directly.
+ HDFS-2246. Shortcut a local client reads to a Datanodes files directly.
(Andrew Purtell, Suresh, Jitendra)
- HADOOP-7804. Enable hadoop config generator to set configurations to enable
+ HADOOP-7804. Enable hadoop config generator to set configurations to enable
short circuit read. (Arpit Gupta via jitendra)
HDFS-2604. Add a log message to show if WebHDFS is enabled and a
@@ -58,17 +58,17 @@ Release 1.0.0 - 2011.11.27
BUG FIXES
- HADOOP-7869. HADOOP_HOME warning happens all of the time (Owen O'Malley
+ HADOOP-7869. HADOOP_HOME warning happens all of the time (Owen O'Malley
via mattf)
- HADOOP-7815. Fixed configuring map memory mb in hadoop-setup-conf.sh.
+ HADOOP-7815. Fixed configuring map memory mb in hadoop-setup-conf.sh.
(Ramya Sunil)
-
- HDFS-2346. TestHost2NodesMap & TestReplicasMap will fail depending upon
+
+ HDFS-2346. TestHost2NodesMap & TestReplicasMap will fail depending upon
execution order of test methods. (Laxman and Uma Maheswara Rao via Matt Foley)
- MAPREDUCE-3374. src/c++/task-controller/configure is not set executable in
- the tarball and that prevents task-controller from rebuilding.
+ MAPREDUCE-3374. src/c++/task-controller/configure is not set executable in
+ the tarball and that prevents task-controller from rebuilding.
(Roman Shaposhnik via Matt Foley)
HDFS-1943. Fail to start datanode while start-dfs.sh is executed by root
@@ -76,7 +76,7 @@ Release 1.0.0 - 2011.11.27
HADOOP-7784. Fixed jsvc packaging. (Eric Yang)
- HADOOP-7740. Fixed security audit logger configuration.
+ HADOOP-7740. Fixed security audit logger configuration.
(Arpit Gupta via Eric Yang)
HADOOP-7765. Clean packaging working directory for Debian packaging.
@@ -124,17 +124,17 @@ Release 1.0.0 - 2011.11.27
HDFS-2528. WebHDFS: set delegation kind to WEBHDFS and add a HDFS token
when http requests are redirected to datanode. (szetszwo)
- HDFS-2540. WebHDFS: change "Expect: 100-continue" to two-step write; change
+ HDFS-2540. WebHDFS: change "Expect: 100-continue" to two-step write; change
"HdfsFileStatus" and "localName" respectively to "FileStatus" and
"pathSuffix" in JSON response. (szetszwo)
- HDFS-1257. Race condition on FSNamesystem#recentInvalidateSets introduced
+ HDFS-1257. Race condition on FSNamesystem#recentInvalidateSets introduced
by HADOOP-5124. (Eric Payne via jitendra)
HDFS-611. Heartbeats times from Datanodes increase when there are plenty of
blocks to delete. (Zheng Shao via jitendra)
- HADOOP-7853. multiple javax security configurations cause conflicts.
+ HADOOP-7853. multiple javax security configurations cause conflicts.
(Daryn via jitendra)
HDFS-2590. Fix the missing links in the WebHDFS forrest doc. (szetszwo)
@@ -145,9 +145,10 @@ Release 1.0.0 - 2011.11.27
MAPREDUCE-3480. Disable TestJvmReuse in branch-1. (jitendra)
- HADOOP-7855. fix to remove datanode dir creation and attribute
+ HADOOP-7855. fix to remove datanode dir creation and attribute
setup from hadoop-conf-setup.sh (gkesavan)
+ HADOOP-7461. Fix to add jackson dependency to hadoop pom. (gkesavan)
Release 0.20.205.0 - 2011.10.06
@@ -158,11 +159,11 @@ Release 0.20.205.0 - 2011.10.06
HDFS-200. Support append and sync for hadoop 0.20 branch. (dhruba)
- HDFS-826. Allow a mechanism for an application to detect that
+ HDFS-826. Allow a mechanism for an application to detect that
datanode(s) have died in the write pipeline. (dhruba)
HDFS-142. Blocks that are being written by a client are stored in the
- blocksBeingWritten directory.
+ blocksBeingWritten directory.
(Dhruba Borthakur, Nicolas Spiegelberg, Todd Lipcon via dhruba)
HDFS-630. Client can exclude specific nodes in the write pipeline.
@@ -171,13 +172,13 @@ Release 0.20.205.0 - 2011.10.06
HDFS-895. Allow hflush/sync to occur in parallel with new writes to
the file. (Todd Lipcon via hairong)
- HDFS-1520. Lightweight NameNode operation recoverLease to trigger
+ HDFS-1520. Lightweight NameNode operation recoverLease to trigger
lease recovery. (Hairong Kuang via dhruba)
MAPREDUCE-2764. Allow JobTracker to renew and cancel arbitrary token types,
including delegation tokens obtained via hftp. (omalley)
- HADOOP-7119 add Kerberos HTTP SPNEGO authentication support to
+ HADOOP-7119 add Kerberos HTTP SPNEGO authentication support to
Hadoop JT/NN/DN/TT web-consoles backport from Trunk (sanjay)
HDFS-2284. Add a new FileSystem, webhdfs://, for supporting write Http
@@ -200,12 +201,12 @@ Release 0.20.205.0 - 2011.10.06
HDFS-2385. Support renew and cancel delegation tokens in WebHDFS.
(szetszwo)
- MAPREDUCE-2777. Backport of MAPREDUCE-220 and MAPREDUCE-2469. Includes adding
+ MAPREDUCE-2777. Backport of MAPREDUCE-220 and MAPREDUCE-2469. Includes adding
cumulative CPU usage and total heap usage to task conters. (amarrk)
BUG FIXES
- HDFS-2404. WebHDFS liststatus json response is not correct.
+ HDFS-2404. WebHDFS liststatus json response is not correct.
(Suresh Srinivas via mattf)
HDFS-2358. NPE when the default filesystem's uri has no authority.
@@ -216,7 +217,7 @@ Release 0.20.205.0 - 2011.10.06
HADOOP-7691. Hadoop deb pkg group id. (Eric Yang via mattf)
- HADOOP-7685. Resolve issues with hadoop-common file hadoop-setup-conf.sh.
+ HADOOP-7685. Resolve issues with hadoop-common file hadoop-setup-conf.sh.
(Eric Yang and Devaraj K, via mattf)
HADOOP-7684. jobhistory server and secondarynamenode should have init.d script
@@ -225,10 +226,10 @@ Release 0.20.205.0 - 2011.10.06
HADOOP-7683. remove hdfs-site.xml template has properties that are not used
in 0.20-security. (Arpit Gupta via mattf)
- HADOOP-7603. Set default hdfs, mapred uid, and hadoop group gid for RPM
+ HADOOP-7603. Set default hdfs, mapred uid, and hadoop group gid for RPM
packages. (Eric Yang via mattf)
- HADOOP-7681. log4j.properties is missing properties for security audit and
+ HADOOP-7681. log4j.properties is missing properties for security audit and
hdfs audit should be changed to info. (Arpit Gupta via mattf)
HADOOP-7679. log4j.properties templates must define mapred.jobsummary.logger
@@ -242,18 +243,18 @@ Release 0.20.205.0 - 2011.10.06
MAPREDUCE-2324. Removed usage of broken
ResourceEstimator.getEstimatedReduceInputSize to check against usable
- disk-space on TaskTracker. (Robert Evans via acmurthy)
+ disk-space on TaskTracker. (Robert Evans via acmurthy)
MAPREDUCE-2729. Ensure jobs with reduces which can't be launched due to
- slow-start do not count for user-limits. (Sherry Chen via acmurthy)
+ slow-start do not count for user-limits. (Sherry Chen via acmurthy)
HADOOP-6833. IPC leaks call parameters when exceptions thrown.
(Todd Lipcon via eli)
-
- HADOOP-7400. Fix HdfsProxyTests fails when the -Dtest.build.dir
+
+ HADOOP-7400. Fix HdfsProxyTests fails when the -Dtest.build.dir
and -Dbuild.test is set a dir other than build dir (gkesavan)
- MAPREDUCE-2650. back-port MAPREDUCE-2238 to 0.20-security.
+ MAPREDUCE-2650. back-port MAPREDUCE-2238 to 0.20-security.
(Sherry Chen via mahadev)
HDFS-2053. Bug in INodeDirectory#computeContentSummary warning
@@ -261,8 +262,8 @@ Release 0.20.205.0 - 2011.10.06
HDFS-2117. DiskChecker#mkdirsWithExistsAndPermissionCheck may
return true even when the dir is not created. (eli)
-
- MAPREDUCE-2489. Jobsplits with random hostnames can make the
+
+ MAPREDUCE-2489. Jobsplits with random hostnames can make the
queue unusable. (Jeffrey Naisbitt via mahadev)
HDFS-2190. NN fails to start if it encounters an empty or malformed fstime
@@ -276,7 +277,7 @@ Release 0.20.205.0 - 2011.10.06
HDFS-606. Fix ConcurrentModificationException in invalidateCorruptReplicas.
(Todd Lipcon via dhruba)
- HDFS-1118. Fix socketleak on DFSClient.
+ HDFS-1118. Fix socketleak on DFSClient.
(Zheng Shao via dhruba)
HDFS-988. Fix bug where savenameSpace can corrupt edits log.
@@ -291,12 +292,12 @@ Release 0.20.205.0 - 2011.10.06
HDFS-1141. completeFile does not check lease ownership.
(Todd Lipcon via dhruba)
- HDFS-1204. Lease expiration should recover single files,
+ HDFS-1204. Lease expiration should recover single files,
not entire lease holder (Sam Rash via dhruba)
HDFS-1346. DFSClient receives out of order packet ack. (hairong)
- HDFS-1057. Concurrent readers hit ChecksumExceptions if following
+ HDFS-1057. Concurrent readers hit ChecksumExceptions if following
a writer to very end of file (Sam Rash via dhruba)
HDFS-724. Use a bidirectional heartbeat to detect stuck
@@ -308,25 +309,25 @@ Release 0.20.205.0 - 2011.10.06
HDFS-1554. New semantics for recoverLease. (hairong)
HADOOP-7596. Makes packaging of 64-bit jsvc possible. Has other
- bug fixes to do with packaging. (Eric Yang via ddas)
+ bug fixes to do with packaging. (Eric Yang via ddas)
HDFS-2309. TestRenameWhileOpen fails. (jitendra)
HDFS-2300. TestFileAppend4 and TestMultiThreadedSync failure. (jitendra)
- HDFS-1122. client block verification may result in blocks in
+ HDFS-1122. client block verification may result in blocks in
DataBlockScanner prematurely. (Sam Rash via jitendra)
- HADOOP-6722. NetUtils.connect should check that it hasn't connected a
+ HADOOP-6722. NetUtils.connect should check that it hasn't connected a
socket to itself. (Todd Lipcon via suresh)
- HDFS-1779. After NameNode restart , Clients can not read partial files
+ HDFS-1779. After NameNode restart , Clients can not read partial files
even after client invokes Sync. (Uma Maheswara Rao G via jitendra)
- HDFS-1197. Blocks are considered "complete" prematurely after
+ HDFS-1197. Blocks are considered "complete" prematurely after
commitBlockSynchronization or DN restart. (Todd Lipcon via jitendra)
- HDFS-1218. Blocks recovered on startup should be treated with lower
+ HDFS-1218. Blocks recovered on startup should be treated with lower
priority during block synchronization. (Todd Lipcon via suresh)
HDFS-1186. DNs should interrupt writers at start of recovery.
@@ -341,9 +342,9 @@ Release 0.20.205.0 - 2011.10.06
HADOOP-7626. Bugfix for a config generator (Eric Yang via ddas)
MAPREDUCE-2549. Fix resource leaks in Eclipse plugin. (Devaraj K via
- acmurthy)
+ acmurthy)
- HDFS-2328. HFTP throws NPE if security is enabled locally, but not
+ HDFS-2328. HFTP throws NPE if security is enabled locally, but not
remotely. (omalley)
HADOOP-7602. wordcount, sort etc on har files fails with NPE.
@@ -355,10 +356,10 @@ Release 0.20.205.0 - 2011.10.06
HADOOP-7644. Fix TestDelegationTokenRenewal and TestDelegationTokenFetcher
to use and test the new style renewers. (omalley)
- HADOOP-7637. Fix to include FairScheduler configuration file in
+ HADOOP-7637. Fix to include FairScheduler configuration file in
RPM. (Eric Yang via ddas)
- HADOOP-7633. Adds log4j.properties to the hadoop-conf dir on
+ HADOOP-7633. Adds log4j.properties to the hadoop-conf dir on
deploy (Eric Yang via ddas)
HADOOP-7631. Fixes a config problem to do with running streaming jobs
@@ -368,12 +369,12 @@ Release 0.20.205.0 - 2011.10.06
*.period set to a default value. (Eric Yang via ddas)
HADOOP-7615. Fixes to have contrib jars in the HADOOP_CLASSPATH
- for the binary layout case. (Eric Yang via ddas)
+ for the binary layout case. (Eric Yang via ddas)
- HADOOP-7661. FileSystem.getCanonicalServiceName throws NPE for any
+ HADOOP-7661. FileSystem.getCanonicalServiceName throws NPE for any
file system uri that doesn't have an authority. (jitendra)
- HADOOP-7649. TestMapredGroupMappingServiceRefresh and
+ HADOOP-7649. TestMapredGroupMappingServiceRefresh and
TestRefreshUserMappings fail after HADOOP-7625. (jitendra)
HADOOP-7658. Fix hadoop config template for secured and unsecured
@@ -408,7 +409,7 @@ Release 0.20.205.0 - 2011.10.06
HDFS-2375. Fix TestFileAppend4 failure. (suresh)
- HDFS-2373. Commands using WebHDFS and hftp print unnecessary debug
+ HDFS-2373. Commands using WebHDFS and hftp print unnecessary debug
info on the console with security enabled. (Arpit Gupta via suresh)
HADOOP-7610. Fix for hadoop debian package (Eric Yang via gkesavan)
@@ -423,7 +424,7 @@ Release 0.20.205.0 - 2011.10.06
HDFS-2408. DFSClient#getNumCurrentReplicas is package private in 205 but
public in branch-0.20-append (stack via atm)
- HADOOP-7721. dfs.web.authentication.kerberos.principal expects the full
+ HADOOP-7721. dfs.web.authentication.kerberos.principal expects the full
hostname and does not replace _HOST with the hostname. (jitendra)
HDFS-2403. NamenodeWebHdfsMethods.generateDelegationToken(..) does not use
@@ -432,7 +433,7 @@ Release 0.20.205.0 - 2011.10.06
HADOOP-7724. Fixed hadoop-setup-conf.sh to put proxy user in
core-site.xml. (Arpit Gupta via Eric Yang)
- HDFS-2411. With WebHDFS enabled in secure mode the auth to local mappings
+ HDFS-2411. With WebHDFS enabled in secure mode the auth to local mappings
are not being respected. (jitendra)
IMPROVEMENTS
@@ -443,7 +444,7 @@ Release 0.20.205.0 - 2011.10.06
cluster. (Arpit Gupta via mattf)
MAPREDUCE-2187. Reporter sends progress during sort/merge. (Anupam Seth via
- acmurthy)
+ acmurthy)
MAPREDUCE-2705. Implements launch of multiple tasks concurrently.
(Thomas Graves via ddas)
@@ -460,19 +461,19 @@ Release 0.20.205.0 - 2011.10.06
HADOOP-7314. Add support for throwing UnknownHostException when a host
doesn't resolve. Needed for MAPREDUCE-2489. (Jeffrey Naisbitt via mattf)
- MAPREDUCE-2494. Make the distributed cache delete entires using LRU
+ MAPREDUCE-2494. Make the distributed cache delete entires using LRU
priority (Robert Joseph Evans via mahadev)
- HADOOP-6889. Make RPC to have an option to timeout - backport to
+ HADOOP-6889. Make RPC to have an option to timeout - backport to
0.20-security. Unit tests updated to 17/Aug/2011 version.
(John George and Ravi Prakash via mattf)
- MAPREDUCE-2780. Use a utility method to set service in token.
+ MAPREDUCE-2780. Use a utility method to set service in token.
(Daryn Sharp via jitendra)
HADOOP-7472. RPC client should deal with IP address change.
(Kihwal Lee via suresh)
-
+
MAPREDUCE-2489. Jobsplits with random hostnames can make the queue unusable
(Jeffrey Naisbit via mahadev)
@@ -480,16 +481,16 @@ Release 0.20.205.0 - 2011.10.06
HDFS-1210. DFSClient should log exception when block recovery fails.
(Todd Lipcon via dhruba)
-
+
HDFS-1211. Block receiver should not log "rewind" packets at INFO level.
(Todd Lipcon)
HDFS-1164. TestHdfsProxy is failing. (Todd Lipcon)
- HDFS-1202. DataBlockScanner throws NPE when updated before initialized.
+ HDFS-1202. DataBlockScanner throws NPE when updated before initialized.
(Todd Lipcon)
- HADOOP-7539. merge hadoop archive goodness from trunk to .20 (John George
+ HADOOP-7539. merge hadoop archive goodness from trunk to .20 (John George
via mahadev)
HADOOP-7594. Support HTTP REST in HttpServer. (szetszwo)
@@ -500,16 +501,16 @@ Release 0.20.205.0 - 2011.10.06
HDFS-2320. Make 0.20-append protocol changes compatible with
0.20-secuirty. (suresh)
- MAPREDUCE-2610. Make QueueAclsInfo public. (Joep Rottinghuis via acmurthy)
+ MAPREDUCE-2610. Make QueueAclsInfo public. (Joep Rottinghuis via acmurthy)
MAPREDUCE-2915. Ensure LTC passes java.library.path. (Kihwal Lee via
- acmurthy)
+ acmurthy)
- HADOOP-7599. Script improvements to setup a secure Hadoop cluster
+ HADOOP-7599. Script improvements to setup a secure Hadoop cluster
(Eric Yang via ddas)
MAPREDUCE-2981. Backport FairScheduler from trunk. (Matei Zaharia via
- acmurthy)
+ acmurthy)
MAPREDUCE-1734. Undeprecate old API in branch-0.20-security. (Todd Lipcon
via acmurthy)
@@ -523,7 +524,7 @@ Release 0.20.205.0 - 2011.10.06
HDFS-2368. Move SPNEGO conf properties from hdfs-default.xml to
hdfs-site.xml. (szetszwo)
- HADOOP-7710. Added hadoop-setup-application.sh for creating
+ HADOOP-7710. Added hadoop-setup-application.sh for creating
application directory (Arpit Gupta via Eric Yang)
HADOOP-7708. Fixed hadoop-setup-conf.sh to handle config file
@@ -560,19 +561,19 @@ Release 0.20.204.0 - 2011-8-25
MAPREDUCE-2651. Fix race condition in Linux task controller for
job log directory creation. (Bharath Mundlapudi via llu)
- MAPREDUCE-2621. TestCapacityScheduler fails with "Queue "q1" does not
+ MAPREDUCE-2621. TestCapacityScheduler fails with "Queue "q1" does not
exist". (Sherry Chen via mahadev)
HADOOP-7475. Fix hadoop-setup-single-node.sh to reflect new layout. (eyang
via omalley)
- HADOOP-7045. TestDU fails on systems with local file systems with
+ HADOOP-7045. TestDU fails on systems with local file systems with
extended attributes. (eli)
MAPREDUCE-2495. exit() the TaskTracker when the distributed cache cleanup
thread dies. (Robert Joseph Evans via cdouglas)
- HDFS-1878. TestHDFSServerPorts unit test failure - race condition
+ HDFS-1878. TestHDFSServerPorts unit test failure - race condition
in FSNamesystem.close() causes NullPointerException without serious
consequence. (mattf)
@@ -592,7 +593,7 @@ Release 0.20.204.0 - 2011-8-25
failed fetch notifications in the JobTracker log.
(Jeffrey Naisbitt via cdouglas)
- HDFS-2044. TestQueueProcessingStatistics failing automatic test due to
+ HDFS-2044. TestQueueProcessingStatistics failing automatic test due to
timing issues. (mattf)
HADOOP-7248. Update eclipse target to generate .classpath from ivy config.
@@ -600,8 +601,8 @@ Release 0.20.204.0 - 2011-8-25
MAPREDUCE-2558. Add queue-level metrics 0.20-security branch (test fixes)
(Jeffrey Naisbitt via mahadev)
-
- HADOOP-7364. TestMiniMRDFSCaching fails if test.build.dir is set to
+
+ HADOOP-7364. TestMiniMRDFSCaching fails if test.build.dir is set to
something other than build/test. (Thomas Graves via mahadev)
HADOOP-7277. Add generation of run configurations to eclipse target.
@@ -631,13 +632,13 @@ Release 0.20.204.0 - 2011-8-25
diagnosis of related issues. (Jonathan Eagles via cdouglas)
MAPREDUCE-2447. Fix Child.java to set Task.jvmContext sooner to avoid
- corner cases in error handling. (Siddharth Seth via acmurthy)
+ corner cases in error handling. (Siddharth Seth via acmurthy)
MAPREDUCE-2429. Validate JVM in TaskUmbilicalProtocol. (Siddharth Seth via
- acmurthy)
+ acmurthy)
MAPREDUCE-2418. Show job errors in JobHistory page. (Siddharth Seth via
- acmurthy)
+ acmurthy)
HDFS-1592. At Startup, Valid volumes required in FSDataset doesn't
handle consistently with volumes tolerated. (Bharath Mundlapudi)
@@ -657,22 +658,22 @@ Release 0.20.204.0 - 2011-8-25
MAPREDUCE-2409. Distinguish distributed cache artifacts localized as
files, archives. (Siddharth Seth via cdouglas)
- MAPREDUCE-118. Fix Job.getJobID() to get the new ID as soon as it's
+ MAPREDUCE-118. Fix Job.getJobID() to get the new ID as soon as it's
assigned. (Amareshwari Sriramadasu and Dick King via cdouglas)
MAPREDUCE-2411. Force an exception when the queue has an invalid name or
its ACLs are misconfigured. (Dick King via cdouglas)
- HDFS-1258. Clearing namespace quota on "/" corrupts fs image.
+ HDFS-1258. Clearing namespace quota on "/" corrupts fs image.
(Aaron T. Myers via szetszwo)
HDFS-1189. Quota counts missed between clear quota and set quota.
(John George via szetszwo)
- HDFS-1692. In secure mode, Datanode process doesn't exit when disks
+ HDFS-1692. In secure mode, Datanode process doesn't exit when disks
fail. (bharathm via boryas)
- MAPREDUCE-2420. JobTracker should be able to renew delegation token
+ MAPREDUCE-2420. JobTracker should be able to renew delegation token
over HTTP (boryas)
MAPREDUCE-2443. Fix TaskAspect for TaskUmbilicalProtocol.ping(..).
@@ -687,9 +688,9 @@ Release 0.20.204.0 - 2011-8-25
HDFS-2057. Wait time to terminate the threads causes unit tests to
take longer time. (Bharath Mundlapudi via suresh)
- HDFS-2218. Disable TestHdfsProxy.testHdfsProxyInterface in automated test
+ HDFS-2218. Disable TestHdfsProxy.testHdfsProxyInterface in automated test
suite for 0.20-security-204 release. (Matt Foley)
-
+
IMPROVEMENTS
HADOOP-7144. Expose JMX metrics via JSON servlet. (Robert Joseph Evans via
@@ -711,7 +712,7 @@ Release 0.20.204.0 - 2011-8-25
MAPREDUCE-2415. Distribute the user task logs on to multiple disks.
(Bharath Mundlapudi via omalley)
- MAPREDUCE-2413. TaskTracker should handle disk failures by reinitializing
+ MAPREDUCE-2413. TaskTracker should handle disk failures by reinitializing
itself. (Ravi Gummadi and Jagane Sundar via omalley)
HDFS-1541. Not marking datanodes dead when namenode in safemode.
@@ -724,7 +725,7 @@ Release 0.20.204.0 - 2011-8-25
HADOOP-7459. Remove jdk-1.6.0 dependency check from rpm. (omalley)
- HADOOP-7330. Fix MetricsSourceAdapter to use the value instead of the
+ HADOOP-7330. Fix MetricsSourceAdapter to use the value instead of the
object. (Luke Lu via omalley)
Release 0.20.203.0 - 2011-5-11
@@ -745,46 +746,46 @@ Release 0.20.203.0 - 2011-5-11
HADOOP-7143. Restore HadoopArchives. (Joep Rottinghuis via omalley)
- MAPREDUCE-2316. Updated CapacityScheduler documentation. (acmurthy)
+ MAPREDUCE-2316. Updated CapacityScheduler documentation. (acmurthy)
HADOOP-7243. Fix contrib unit tests missing dependencies. (omalley)
HADOOP-7190. Add metrics v1 back for backwards compatibility. (omalley)
- MAPREDUCE-2360. Remove stripping of scheme, authority from submit dir in
+ MAPREDUCE-2360. Remove stripping of scheme, authority from submit dir in
support of viewfs. (cdouglas)
-
+
MAPREDUCE-2359 Use correct file system to access distributed cache objects.
(Krishna Ramachandran)
-
+
MAPREDUCE-2361. "Fix Distributed Cache is not adding files to class paths
correctly" - Drop the host/scheme/fragment from URI (cdouglas)
MAPREDUCE-2362. Fix unit-test failures: TestBadRecords (NPE due to
rearranged MapTask code) and TestTaskTrackerMemoryManager
- (need hostname in output-string pattern). (Greg Roelofs, Krishna
+ (need hostname in output-string pattern). (Greg Roelofs, Krishna
Ramachandran)
HDFS-1729. Add statistics logging for better visibility into
startup time costs. (Matt Foley)
- MAPREDUCE-2363. When a queue is built without any access rights we
+ MAPREDUCE-2363. When a queue is built without any access rights we
explain the problem. (Richard King)
- MAPREDUCE-1563. TaskDiagnosticInfo may be missed sometime. (Krishna
+ MAPREDUCE-1563. TaskDiagnosticInfo may be missed sometime. (Krishna
Ramachandran)
MAPREDUCE-2364. Don't hold the rjob lock while localizing resources. (ddas
via omalley)
- MAPREDUCE-2365. New counters for FileInputFormat (BYTES_READ) and
- FileOutputFormat (BYTES_WRITTEN).
+ MAPREDUCE-2365. New counters for FileInputFormat (BYTES_READ) and
+ FileOutputFormat (BYTES_WRITTEN).
New counter MAP_OUTPUT_MATERIALIZED_BYTES for compressed MapOutputSize.
(Siddharth Seth)
HADOOP-7040. Change DiskErrorException to IOException (boryas)
- HADOOP-7104. Remove unnecessary DNS reverse lookups from RPC layer
+ HADOOP-7104. Remove unnecessary DNS reverse lookups from RPC layer
(kzhang)
MAPREDUCE-2366. Fix a problem where the task browser UI can't retrieve the
@@ -799,7 +800,7 @@ Release 0.20.203.0 - 2011-5-11
HADOOP-6879. Provide SSH based (Jsch) remote execution API for system
tests. (cos)
- HADOOP-7215. RPC clients must use network interface corresponding to
+ HADOOP-7215. RPC clients must use network interface corresponding to
the host in the client's kerberos principal key. (suresh)
HADOOP-7232. Fix Javadoc warnings. (omalley)
@@ -808,14 +809,14 @@ Release 0.20.203.0 - 2011-5-11
Release 0.20.202.0 - unreleased
- MAPREDUCE-2355. Add a configuration knob
+ MAPREDUCE-2355. Add a configuration knob
mapreduce.tasktracker.outofband.heartbeat.damper that limits out of band
heartbeats (acmurthy)
- MAPREDUCE-2356. Fix a race-condition that corrupted a task's state on the
+ MAPREDUCE-2356. Fix a race-condition that corrupted a task's state on the
JobTracker. (Luke Lu)
- MAPREDUCE-2357. Always propagate IOExceptions that are thrown by
+ MAPREDUCE-2357. Always propagate IOExceptions that are thrown by
non-FileInputFormat. (Luke Lu)
HADOOP-7163. RPC handles SocketTimeOutException during SASL negotiation.
@@ -824,15 +825,15 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-2358. MapReduce assumes the default FileSystem is HDFS.
(Krishna Ramachandran)
- MAPREDUCE-1904. Reducing locking contention in TaskTracker's
+ MAPREDUCE-1904. Reducing locking contention in TaskTracker's
MapOutputServlet LocalDirAllocator. (Rajesh Balamohan via acmurthy)
HDFS-1626. Make BLOCK_INVALIDATE_LIMIT configurable. (szetszwo)
- HDFS-1584. Adds a check for whether relogin is needed to
+ HDFS-1584. Adds a check for whether relogin is needed to
getDelegationToken in HftpFileSystem. (Kan Zhang via ddas)
- HADOOP-7115. Reduces the number of calls to getpwuid_r and
+ HADOOP-7115. Reduces the number of calls to getpwuid_r and
getpwgid_r, by implementing a cache in NativeIO. (ddas)
HADOOP-6882. An XSS security exploit in jetty-6.1.14. jetty upgraded to
@@ -843,15 +844,15 @@ Release 0.20.202.0 - unreleased
HDFS-1353 redux. Modulate original 1353 to not bump RPC version.
(jhoman)
- MAPREDUCE-2082 Race condition in writing the jobtoken password file when
+ MAPREDUCE-2082 Race condition in writing the jobtoken password file when
launching pipes jobs (jitendra and ddas)
-
- HADOOP-6978. Fixes task log servlet vulnerabilities via symlinks.
+
+ HADOOP-6978. Fixes task log servlet vulnerabilities via symlinks.
(Todd Lipcon and Devaraj Das)
- MAPREDUCE-2178. Write task initialization to avoid race
- conditions leading to privilege escalation and resource leakage by
- performing more actiions as the user. (Owen O'Malley, Devaraj Das,
+ MAPREDUCE-2178. Write task initialization to avoid race
+ conditions leading to privilege escalation and resource leakage by
+ performing more actiions as the user. (Owen O'Malley, Devaraj Das,
Chris Douglas via cdouglas)
HDFS-1364. HFTP client should support relogin from keytab
@@ -859,16 +860,16 @@ Release 0.20.202.0 - unreleased
HADOOP-6907. Make RPC client to use per-proxy configuration.
(Kan Zhang via ddas)
- MAPREDUCE-2055. Fix JobTracker to decouple job retirement from copy of
- job-history file to HDFS and enhance RetiredJobInfo to carry aggregated
- job-counters to prevent a disk roundtrip on job-completion to fetch
+ MAPREDUCE-2055. Fix JobTracker to decouple job retirement from copy of
+ job-history file to HDFS and enhance RetiredJobInfo to carry aggregated
+ job-counters to prevent a disk roundtrip on job-completion to fetch
counters for the JobClient. (Krishna Ramachandran via acmurthy)
-
+
HDFS-1353. Remove most of getBlockLocation optimization (jghoman)
MAPREDUCE-2023. TestDFSIO read test may not read specified bytes. (htang)
- HDFS-1340. A null delegation token is appended to the url if security is
+ HDFS-1340. A null delegation token is appended to the url if security is
disabled when browsing filesystem.(boryas)
HDFS-1352. Fix jsvc.location. (jghoman)
@@ -886,7 +887,7 @@ Release 0.20.202.0 - unreleased
HADOOP-6899. RawLocalFileSystem set working directory does
not work for relative names. (suresh)
- HDFS-495. New clients should be able to take over files lease if the old
+ HDFS-495. New clients should be able to take over files lease if the old
client died. (shv)
HADOOP-6728. Re-design and overhaul of the Metrics framework. (Luke Lu via
@@ -899,13 +900,13 @@ Release 0.20.202.0 - unreleased
HADOOP-6864. Add ability to get netgroups (as returned by getent
netgroup command) using native code (JNI) instead of forking. (Erik Steffl)
- HDFS-1318. HDFS Namenode and Datanode WebUI information needs to be
+ HDFS-1318. HDFS Namenode and Datanode WebUI information needs to be
accessible programmatically for scripts. (Tanping Wang via suresh)
- HDFS-1315. Add fsck event to audit log and remove other audit log events
+ HDFS-1315. Add fsck event to audit log and remove other audit log events
corresponding to FSCK listStatus and open calls. (suresh)
- MAPREDUCE-1941. Provides access to JobHistory file (raw) with job user/acl
+ MAPREDUCE-1941. Provides access to JobHistory file (raw) with job user/acl
permission. (Srikanth Sundarrajan via ddas)
MAPREDUCE-291. Optionally a separate daemon should serve JobHistory.
@@ -929,34 +930,34 @@ Release 0.20.202.0 - unreleased
HDFS-457. Reintroduce volume failure tolerance for DataNodes. (shv)
- HDFS-1307 Add start time, end time and total time taken for FSCK
+ HDFS-1307 Add start time, end time and total time taken for FSCK
to FSCK report. (suresh)
- MAPREDUCE-1207. Sanitize user environment of map/reduce tasks and allow
+ MAPREDUCE-1207. Sanitize user environment of map/reduce tasks and allow
admins to set environment and java options. (Krishna Ramachandran via
- acmurthy)
+ acmurthy)
HDFS-1298 - Add support in HDFS for new statistics added in FileSystem
to track the file system operations (suresh)
- HDFS-1301. TestHDFSProxy need to use server side conf for ProxyUser
+ HDFS-1301. TestHDFSProxy need to use server side conf for ProxyUser
stuff.(boryas)
- HADOOP-6859 - Introduce additional statistics to FileSystem to track
+ HADOOP-6859 - Introduce additional statistics to FileSystem to track
file system operations (suresh)
- HADOOP-6818. Provides a JNI implementation of Unix Group resolution. The
- config hadoop.security.group.mapping should be set to
+ HADOOP-6818. Provides a JNI implementation of Unix Group resolution. The
+ config hadoop.security.group.mapping should be set to
org.apache.hadoop.security.JniBasedUnixGroupsMapping to enable this
implementation. (ddas)
MAPREDUCE-1938. Introduces a configuration for putting user classes before
the system classes during job submission and in task launches. Two things
- need to be done in order to use this feature -
- (1) mapreduce.user.classpath.first : this should be set to true in the
- jobconf, and, (2) HADOOP_USER_CLASSPATH_FIRST : this is relevant for job
+ need to be done in order to use this feature -
+ (1) mapreduce.user.classpath.first : this should be set to true in the
+ jobconf, and, (2) HADOOP_USER_CLASSPATH_FIRST : this is relevant for job
submissions done using bin/hadoop shell script. HADOOP_USER_CLASSPATH_FIRST
- should be defined in the environment with some non-empty value
+ should be defined in the environment with some non-empty value
(like "true"), and then bin/hadoop should be executed. (ddas)
HADOOP-6669. Respect compression configuration when creating DefaultCodec
@@ -972,23 +973,23 @@ Release 0.20.202.0 - unreleased
done while replaying edits during startup. (shv)
MAPREDUCE-1521. Protection against incorrectly configured reduces
- (mahadev)
+ (mahadev)
MAPREDUCE-1936. Make Gridmix3 more customizable. (htang)
MAPREDUCE-517. Enhance the CapacityScheduler to assign multiple tasks
- per-heartbeat. (acmurthy)
+ per-heartbeat. (acmurthy)
- MAPREDUCE-323. Re-factor layout of JobHistory files on HDFS to improve
- operability. (Dick King via acmurthy)
+ MAPREDUCE-323. Re-factor layout of JobHistory files on HDFS to improve
+ operability. (Dick King via acmurthy)
MAPREDUCE-1921. Ensure exceptions during reading of input data in map
tasks are augmented by information about actual input file which caused
- the exception. (Krishna Ramachandran via acmurthy)
+ the exception. (Krishna Ramachandran via acmurthy)
MAPREDUCE-1118. Enhance the JobTracker web-ui to ensure tabular columns
are sortable, also added a /scheduler servlet to CapacityScheduler for
- enhanced UI for queue information. (Krishna Ramachandran via acmurthy)
+ enhanced UI for queue information. (Krishna Ramachandran via acmurthy)
HADOOP-5913. Add support for starting/stopping queues. (cdouglas)
@@ -1010,7 +1011,7 @@ Release 0.20.202.0 - unreleased
HDFS-1250. Namenode should reject block reports and block received
requests from dead datanodes (suresh)
- MAPREDUCE-1863. [Rumen] Null failedMapAttemptCDFs in job traces generated
+ MAPREDUCE-1863. [Rumen] Null failedMapAttemptCDFs in job traces generated
by Rumen. (htang)
MAPREDUCE-1309. Rumen refactory. (htang)
@@ -1024,23 +1025,23 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-787. Fix JobSubmitter to honor user given symlink in the path.
(amareshwari)
- HADOOP-6815. refreshSuperUserGroupsConfiguration should use
+ HADOOP-6815. refreshSuperUserGroupsConfiguration should use
server side configuration for the refresh( boryas)
MAPREDUCE-1868. Add a read and connection timeout to JobClient while
- pulling tasklogs. (Krishna Ramachandran via acmurthy)
+ pulling tasklogs. (Krishna Ramachandran via acmurthy)
HDFS-1119. Introduce a GSet interface to BlocksMap. (szetszwo)
MAPREDUCE-1778. Ensure failure to setup CompletedJobStatusStore is not
- silently ignored by the JobTracker. (Krishna Ramachandran via acmurthy)
+ silently ignored by the JobTracker. (Krishna Ramachandran via acmurthy)
MAPREDUCE-1538. Add a limit on the number of artifacts in the
DistributedCache to ensure we cleanup aggressively. (Dick King via
- acmurthy)
+ acmurthy)
MAPREDUCE-1850. Add information about the host from which a job is
- submitted. (Krishna Ramachandran via acmurthy)
+ submitted. (Krishna Ramachandran via acmurthy)
HDFS-1110. Reuses objects for commonly used file names in namenode to
reduce the heap usage. (suresh)
@@ -1070,14 +1071,14 @@ Release 0.20.202.0 - unreleased
improvement. (Balaji Rajagopalan via cos)
MAPREDUCE-1288. Fixes TrackerDistributedCacheManager to take into account
- the owner of the localized file in the mapping from cache URIs to
+ the owner of the localized file in the mapping from cache URIs to
CacheStatus objects. (ddas)
-
+
MAPREDUCE-1682. Fix speculative execution to ensure tasks are not
- scheduled after job failure. (acmurthy)
+ scheduled after job failure. (acmurthy)
MAPREDUCE-1914. Ensure unique sub-directories for artifacts in the
- DistributedCache are cleaned up. (Dick King via acmurthy)
+ DistributedCache are cleaned up. (Dick King via acmurthy)
MAPREDUCE-1538. Add a limit on the number of artifacts in the
DistributedCache to ensure we cleanup aggressively. (Dick King via
@@ -1096,7 +1097,7 @@ Release 0.20.202.0 - unreleased
with handling of null hostname. (omalley)
HDFS-1007. makes long running servers using hftp work. Also has some
- refactoring in the MR code to do with handling of delegation tokens.
+ refactoring in the MR code to do with handling of delegation tokens.
(omalley & ddas)
HDFS-1178. The NameNode servlets should not use RPC to connect to the
@@ -1117,29 +1118,29 @@ Release 0.20.202.0 - unreleased
HDFS-1150. Bugfix in the hadoop shell script. (ddas)
- HDFS-1153. The navigation to /dfsnodelist.jsp with invalid input
+ HDFS-1153. The navigation to /dfsnodelist.jsp with invalid input
parameters produces NPE and HTTP 500 error (rphulari)
-
+
MAPREDUCE-1664. Bugfix to enable queue administrators of a queue to
view job details of jobs submitted to that queue even though they
- are not part of acl-view-job.
+ are not part of acl-view-job.
HDFS-1150. Bugfix to add more knobs to secure datanode starter.
HDFS-1157. Modifications introduced by HDFS-1150 are breaking aspect's
bindings (cos)
- HDFS-1130. Adds a configuration dfs.cluster.administrators for
+ HDFS-1130. Adds a configuration dfs.cluster.administrators for
controlling access to the default servlets in hdfs. (ddas)
- HADOOP-6706.FIX. Relogin behavior for RPC clients could be improved
+ HADOOP-6706.FIX. Relogin behavior for RPC clients could be improved
(boryas)
HDFS-1150. Verify datanodes' identities to clients in secure clusters.
(jghoman)
MAPREDUCE-1442. Fixed regex in job-history related to parsing Counter
- values. (Luke Lu via acmurthy)
+ values. (Luke Lu via acmurthy)
HADOOP-6760. WebServer shouldn't increase port number in case of negative
port setting caused by Jetty's race. (cos)
@@ -1151,7 +1152,7 @@ Release 0.20.202.0 - unreleased
on a SASL connection failure, and retries again with a new
connection. (ddas)
- MAPREDUCE-1716. Fix on top of earlier patch for logs truncation a.k.a
+ MAPREDUCE-1716. Fix on top of earlier patch for logs truncation a.k.a
MAPREDUCE-1100. Addresses log truncation issues when binary data is
written to log files and adds a header to a truncated log file to
inform users of the done trucation.
@@ -1159,14 +1160,14 @@ Release 0.20.202.0 - unreleased
HDFS-1383. Improve the error messages when using hftp://.
MAPREDUCE-1744. Fixed DistributedCache apis to take a user-supplied
- FileSystem to allow for better proxy behaviour for Oozie. (Richard King)
+ FileSystem to allow for better proxy behaviour for Oozie. (Richard King)
- MAPREDUCE-1733. Authentication between pipes processes and java
+ MAPREDUCE-1733. Authentication between pipes processes and java
counterparts. (jitendra)
MAPREDUCE-1664. Bugfix on top of the previous patch. (ddas)
- HDFS-1136. FileChecksumServlets.RedirectServlet doesn't carry forward
+ HDFS-1136. FileChecksumServlets.RedirectServlet doesn't carry forward
the delegation token (boryas)
HADOOP-6756. Change value of FS_DEFAULT_NAME_KEY from fs.defaultFS
@@ -1189,7 +1190,7 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-1317. NPE in setHostName in Rumen. (rksingh)
- MAPREDUCE-1754. Replace mapred.persmissions.supergroup with an acl :
+ MAPREDUCE-1754. Replace mapred.persmissions.supergroup with an acl :
mapreduce.cluster.administrators and HADOOP-6748.: Remove
hadoop.cluster.administrators. Contributed by Amareshwari Sriramadasu.
@@ -1203,7 +1204,7 @@ Release 0.20.202.0 - unreleased
HADOOP-6745. adding some java doc to Server.RpcMetrics, UGI (boryas)
- MAPREDUCE-1707. TaskRunner can get NPE in getting ugi from TaskTracker.
+ MAPREDUCE-1707. TaskRunner can get NPE in getting ugi from TaskTracker.
(vinodkv)
HDFS-1104. Fsck triggers full GC on NameNode. (hairong)
@@ -1227,10 +1228,10 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-1641. Bug-fix to ensure command line options such as
-files/-archives are checked for duplicate artifacts in the
- DistributedCache. (Amareshwari Sreeramadasu via acmurthy)
+ DistributedCache. (Amareshwari Sreeramadasu via acmurthy)
MAPREDUCE-1641. Fix DistributedCache to ensure same files cannot be put in
- both the archives and files sections. (Richard King via acmurthy)
+ both the archives and files sections. (Richard King via acmurthy)
HADOOP-6670. Fixes a testcase issue introduced by the earlier commit
of the HADOOP-6670 patch. (ddas)
@@ -1241,7 +1242,7 @@ Release 0.20.202.0 - unreleased
HADOOP-6674. Fixes the earlier patch to handle pings correctly (ddas).
- MAPREDUCE-1664. Job Acls affect when Queue Acls are set.
+ MAPREDUCE-1664. Job Acls affect when Queue Acls are set.
(Ravi Gummadi via vinodkv)
HADOOP-6718. Fixes a problem to do with clients not closing RPC
@@ -1253,24 +1254,24 @@ Release 0.20.202.0 - unreleased
HADOOP-6670. Use the UserGroupInformation's Subject as the criteria for
equals and hashCode. (omalley)
- HADOOP-6716. System won't start in non-secure mode when kerb5.conf
+ HADOOP-6716. System won't start in non-secure mode when kerb5.conf
(edu.mit.kerberos on Mac) is not present. (boryas)
- MAPREDUCE-1607. Task controller may not set permissions for a
- task cleanup attempt's log directory. (Amareshwari Sreeramadasu via
+ MAPREDUCE-1607. Task controller may not set permissions for a
+ task cleanup attempt's log directory. (Amareshwari Sreeramadasu via
vinodkv)
- MAPREDUCE-1533. JobTracker performance enhancements. (Amar Kamat via
+ MAPREDUCE-1533. JobTracker performance enhancements. (Amar Kamat via
vinodkv)
- MAPREDUCE-1701. AccessControlException while renewing a delegation token
+ MAPREDUCE-1701. AccessControlException while renewing a delegation token
in not correctly handled in the JobTracker. (boryas)
HDFS-481. Incremental patch to fix broken unit test in contrib/hdfsproxy
HADOOP-6706. Fixes a bug in the earlier version of the same patch (ddas)
- HDFS-1096. allow dfsadmin/mradmin refresh of superuser proxy group
+ HDFS-1096. allow dfsadmin/mradmin refresh of superuser proxy group
mappings(boryas).
HDFS-1012. Support for cluster specific path entries in ldap for hdfsproxy
@@ -1279,7 +1280,7 @@ Release 0.20.202.0 - unreleased
HDFS-1011. Improve Logging in HDFSProxy to include cluster name associated
with the request (Srikanth Sundarrajan via Nicholas)
- HDFS-1010. Retrieve group information from UnixUserGroupInformation
+ HDFS-1010. Retrieve group information from UnixUserGroupInformation
instead of LdapEntry (Srikanth Sundarrajan via Nicholas)
HDFS-481. Bug fix - hdfsproxy: Stack overflow + Race conditions
@@ -1291,9 +1292,9 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-1692. Remove TestStreamedMerge from the streaming tests.
(Amareshwari Sriramadasu and Sreekanth Ramakrishnan via vinodkv)
- HDFS-1081. Performance regression in
+ HDFS-1081. Performance regression in
DistributedFileSystem::getFileBlockLocations in secure systems (jhoman)
-
+
MAPREDUCE-1656. JobStory should provide queue info. (htang)
MAPREDUCE-1317. Reducing memory consumption of rumen objects. (htang)
@@ -1302,9 +1303,9 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-1683. Fixed jobtracker web-ui to correctly display heap-usage.
(acmurthy)
-
+
HADOOP-6706. Fixes exception handling for saslConnect. The ideal
- solution is to the Refreshable interface but as Owen noted in
+ solution is to the Refreshable interface but as Owen noted in
HADOOP-6656, it doesn't seem to work as expected. (ddas)
MAPREDUCE-1617. TestBadRecords failed once in our test runs. (Amar
@@ -1323,19 +1324,19 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-1683. Removes JNI calls to get jvm current/max heap usage in
ClusterStatus by default. (acmurthy)
- HADOOP-6687. user object in the subject in UGI should be reused in case
+ HADOOP-6687. user object in the subject in UGI should be reused in case
of a relogin. (jitendra)
- HADOOP-5647. TestJobHistory fails if /tmp/_logs is not writable to.
+ HADOOP-5647. TestJobHistory fails if /tmp/_logs is not writable to.
Testcase should not depend on /tmp. (Ravi Gummadi via vinodkv)
- MAPREDUCE-181. Bug fix for Secure job submission. (Ravi Gummadi via
+ MAPREDUCE-181. Bug fix for Secure job submission. (Ravi Gummadi via
vinodkv)
- MAPREDUCE-1635. ResourceEstimator does not work after MAPREDUCE-842.
+ MAPREDUCE-1635. ResourceEstimator does not work after MAPREDUCE-842.
(Amareshwari Sriramadasu via vinodkv)
- MAPREDUCE-1526. Cache the job related information while submitting the
+ MAPREDUCE-1526. Cache the job related information while submitting the
job. (rksingh)
HADOOP-6674. Turn off SASL checksums for RPCs. (jitendra via omalley)
@@ -1348,17 +1349,17 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-1594. Support for SleepJobs in Gridmix (rksingh)
HDFS-1007. Fix. ServiceName for delegation token for Hftp has hftp
- port and not RPC port.
-
+ port and not RPC port.
+
MAPREDUCE-1376. Support for varied user submissions in Gridmix (rksingh)
- HDFS-1080. SecondaryNameNode image transfer should use the defined
+ HDFS-1080. SecondaryNameNode image transfer should use the defined
http address rather than local ip address (jhoman)
-
+
HADOOP-6661. User document for UserGroupInformation.doAs for secure
impersonation. (jitendra)
- MAPREDUCE-1624. Documents the job credentials and associated details
+ MAPREDUCE-1624. Documents the job credentials and associated details
to do with delegation tokens (ddas)
HDFS-1036. Documentation for fetchdt for forrest (boryas)
HDFS-1039. New patch on top of previous patch. Gets namenode address
@@ -1372,7 +1373,7 @@ Release 0.20.202.0 - unreleased
HADOOP-6649. An error in the previous committed patch. (jitendra)
- HADOOP-6652. ShellBasedUnixGroupsMapping shouldn't have a cache.
+ HADOOP-6652. ShellBasedUnixGroupsMapping shouldn't have a cache.
(ddas)
HADOOP-6649. login object in UGI should be inside the subject
@@ -1387,16 +1388,16 @@ Release 0.20.202.0 - unreleased
HADOOP-6632. Fix on JobTracker to reuse filesystem handles if possible.
(ddas)
- HADOOP-6647. balancer fails with "is not authorized for protocol
+ HADOOP-6647. balancer fails with "is not authorized for protocol
interface NamenodeProtocol" in secure environment (boryas)
- MAPREDUCE-1612. job conf file is not accessible from job history
+ MAPREDUCE-1612. job conf file is not accessible from job history
web page. (Ravi Gummadi via vinodkv)
- MAPREDUCE-1611. Refresh nodes and refresh queues doesnt work with
+ MAPREDUCE-1611. Refresh nodes and refresh queues doesnt work with
service authorization enabled. (Amar Kamat via vinodkv)
- HADOOP-6644. util.Shell getGROUPS_FOR_USER_COMMAND method
+ HADOOP-6644. util.Shell getGROUPS_FOR_USER_COMMAND method
name - should use common naming convention (boryas)
MAPREDUCE-1609. Fixes a problem with localization of job log
@@ -1414,25 +1415,25 @@ Release 0.20.202.0 - unreleased
HADOOP-6634. AccessControlList uses full-principal names to verify acls
causing queue-acls to fail (vinodkv)
-
- HADOOP-6642. Fix javac, javadoc, findbugs warnings. (chrisdo via acmurthy)
- HDFS-1044. Cannot submit mapreduce job from secure client to
+ HADOOP-6642. Fix javac, javadoc, findbugs warnings. (chrisdo via acmurthy)
+
+ HDFS-1044. Cannot submit mapreduce job from secure client to
unsecure sever. (boryas)
- HADOOP-6638. try to relogin in a case of failed RPC connection
- (expired tgt) only in case the subject is loginUser or
+ HADOOP-6638. try to relogin in a case of failed RPC connection
+ (expired tgt) only in case the subject is loginUser or
proxyUgi.realUser. (boryas)
- HADOOP-6632. Support for using different Kerberos keys for different
+ HADOOP-6632. Support for using different Kerberos keys for different
instances of Hadoop services. (jitendra)
- HADOOP-6526. Need mapping from long principal names to local OS
+ HADOOP-6526. Need mapping from long principal names to local OS
user names. (jitendra)
MAPREDUCE-1604. Update Forrest documentation for job authorization
ACLs. (Amareshwari Sriramadasu via yhemanth)
- HDFS-1045. In secure clusters, re-login is necessary for https
+ HDFS-1045. In secure clusters, re-login is necessary for https
clients before opening connections (jhoman)
HADOOP-6603. Addition to original patch to be explicit
@@ -1444,7 +1445,7 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-1606. Fixed occassinal timeout in TestJobACL. (Ravi Gummadi via
acmurthy)
- HADOOP-6633. normalize property names for JT/NN kerberos principal
+ HADOOP-6633. normalize property names for JT/NN kerberos principal
names in configuration. (boryas)
HADOOP-6613. Changes the RPC server so that version is checked first
@@ -1468,10 +1469,10 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-1522. FileInputFormat may use the default FileSystem for the
input path. (Tsz Wo (Nicholas), SZE via cdouglas)
- HDFS-1036. In DelegationTokenFetch pass Configuration object so
+ HDFS-1036. In DelegationTokenFetch pass Configuration object so
getDefaultUri will work correctly.
- HDFS-1038. In nn_browsedfscontent.jsp fetch delegation token only if
+ HDFS-1038. In nn_browsedfscontent.jsp fetch delegation token only if
security is enabled. (jitendra)
HDFS-1036. in DelegationTokenFetch dfs.getURI returns no port (boryas)
@@ -1479,10 +1480,10 @@ Release 0.20.202.0 - unreleased
HADOOP-6598. Verbose logging from the Group class (one more case)
(boryas)
- HADOOP-6627. Bad Connection to FS" message in FSShell should print
+ HADOOP-6627. Bad Connection to FS" message in FSShell should print
message from the exception (boryas)
- HDFS-1033. In secure clusters, NN and SNN should verify that the remote
+ HDFS-1033. In secure clusters, NN and SNN should verify that the remote
principal during image and edits transfer (jhoman)
HDFS-1005. Fixes a bug to do with calling the cross-realm API in Fsck
@@ -1492,7 +1493,7 @@ Release 0.20.202.0 - unreleased
with non-deletable permissions are created within it.
(Amar Kamat via yhemanth)
- HDFS-1007. Fixes bugs to do with 20S cluster talking to 20 over
+ HDFS-1007. Fixes bugs to do with 20S cluster talking to 20 over
hftp (borya)
MAPREDUCE:1566. Fixes bugs in the earlier patch. (ddas)
@@ -1508,8 +1509,8 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-1435. Add test cases to already committed patch for this
jira, synchronizing changes with trunk. (yhemanth)
- HADOOP-6612. Protocols RefreshUserToGroupMappingsProtocol and
- RefreshAuthorizationPolicyProtocol authorization settings thru
+ HADOOP-6612. Protocols RefreshUserToGroupMappingsProtocol and
+ RefreshAuthorizationPolicyProtocol authorization settings thru
KerberosInfo (boryas)
MAPREDUCE-1566. Bugfix for tests on top of the earlier patch. (ddas)
@@ -1522,7 +1523,7 @@ Release 0.20.202.0 - unreleased
HDFS-1023. Update to original patch to fix username (jhoman)
- HDFS-814. Add an api to get the visible length of a
+ HDFS-814. Add an api to get the visible length of a
DFSDataInputStream. (hairong)
HDFS-1023. Allow http server to start as regular user if https
@@ -1559,8 +1560,8 @@ Release 0.20.202.0 - unreleased
HADOOP-6584. Fix javadoc warnings introduced by original HADOOP-6584
patch (jhoman)
-
- HDFS-1017. browsedfs jsp should call JspHelper.getUGI rather than using
+
+ HDFS-1017. browsedfs jsp should call JspHelper.getUGI rather than using
createRemoteUser(). (jhoman)
MAPREDUCE-899. Modified LinuxTaskController to check that task-controller
@@ -1579,7 +1580,7 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-1155. Just pulls out the TestStreamingExitStatus part of the
patch from jira (that went to 0.22). (ddas)
-
+
HADOOP-6600. Fix for branch backport only. Comparing of user should use
equals. (boryas).
@@ -1588,15 +1589,15 @@ Release 0.20.202.0 - unreleased
HDFS-1007. Fixes a bug on top of the earlier patch. (ddas)
- HDFS-1005. Fsck security. Makes it work over kerberized SSL (boryas and
+ HDFS-1005. Fsck security. Makes it work over kerberized SSL (boryas and
jhoman)
HDFS-1007. Makes HFTP and Distcp use kerberized SSL. (ddas)
- MAPREDUCE-1455. Fixes a testcase in the earlier patch.
+ MAPREDUCE-1455. Fixes a testcase in the earlier patch.
(Ravi Gummadi via ddas)
- HDFS-992. Refactors block access token implementation to conform to the
+ HDFS-992. Refactors block access token implementation to conform to the
generic Token interface. (Kan Zhang via ddas)
HADOOP-6584. Adds KrbSSL connector for jetty. (Jakob Homan via ddas)
@@ -1607,28 +1608,28 @@ Release 0.20.202.0 - unreleased
HADOOP-6600,HDFS-1003,MAPREDUCE-1539. mechanism for authorization check
for inter-server protocols(boryas)
- HADOOP-6580,HDFS-993,MAPREDUCE-1516. UGI should contain authentication
+ HADOOP-6580,HDFS-993,MAPREDUCE-1516. UGI should contain authentication
method.
- Namenode and JT should issue a delegation token only for kerberos
+ Namenode and JT should issue a delegation token only for kerberos
authenticated clients. (jitendra)
HDFS-984,HADOOP-6573,MAPREDUCE-1537. Delegation Tokens should be persisted
in Namenode, and corresponding changes in common and mr. (jitendra)
- HDFS-994. Provide methods for obtaining delegation token from Namenode for
- hftp and other uses. Incorporates HADOOP-6594: Update hdfs script to
+ HDFS-994. Provide methods for obtaining delegation token from Namenode for
+ hftp and other uses. Incorporates HADOOP-6594: Update hdfs script to
provide fetchdt tool. (jitendra)
HADOOP-6586. Log authentication and authorization failures and successes
(boryas)
- HDFS-991. Allow use of delegation tokens to authenticate to the
+ HDFS-991. Allow use of delegation tokens to authenticate to the
HDFS servlets. (omalley)
- HADOOP-1849. Add undocumented configuration parameter for per handler
+ HADOOP-1849. Add undocumented configuration parameter for per handler
call queue size in IPC Server. (shv)
-
+
HADOOP-6599. Split existing RpcMetrics with summary in RpcMetrics and
details information in RpcDetailedMetrics. (suresh)
@@ -1641,7 +1642,7 @@ Release 0.20.202.0 - unreleased
as the job submitter. Also, changes the renewal to maintain list of tokens
to renew. (ddas)
- HADOOP-6596. Add a version field to the seialization of the
+ HADOOP-6596. Add a version field to the seialization of the
AbstractDelegationTokenIdentifier. (omalley)
HADOOP-5561. Add javadoc.maxmemory to build.xml to allow larger memory.
@@ -1656,11 +1657,11 @@ Release 0.20.202.0 - unreleased
HDFS-999.Secondary namenode should login using kerberos if security
is configured(boryas)
- MAPREDUCE-1466. Added a private configuration variable
- mapreduce.input.num.files, to store number of input files
+ MAPREDUCE-1466. Added a private configuration variable
+ mapreduce.input.num.files, to store number of input files
being processed by M/R job. (Arun Murthy via yhemanth)
- MAPREDUCE-1403. Save file-sizes of each of the artifacts in
+ MAPREDUCE-1403. Save file-sizes of each of the artifacts in
DistributedCache in the JobConf (Arun Murthy via yhemanth)
HADOOP-6543. Fixes a compilation problem in the original commit. (ddas)
@@ -1668,7 +1669,7 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-1520. Moves a call to setWorkingDirectory in Child to within
a doAs block. (Amareshwari Sriramadasu via ddas)
- HADOOP-6543. Allows secure clients to talk to unsecure clusters.
+ HADOOP-6543. Allows secure clients to talk to unsecure clusters.
(Kan Zhang via ddas)
MAPREDUCE-1505. Delays construction of the job client until it is really
@@ -1682,10 +1683,10 @@ Release 0.20.202.0 - unreleased
HDFS-946. NameNode should not return full path name when lisitng a
diretory or getting the status of a file. (hairong)
- MAPREDUCE-1398. Fix TaskLauncher to stop waiting for slots on a TIP
+ MAPREDUCE-1398. Fix TaskLauncher to stop waiting for slots on a TIP
that is killed / failed. (Amareshwari Sriramadasu via yhemanth)
- MAPREDUCE-1476. Fix the M/R framework to not call commit for special
+ MAPREDUCE-1476. Fix the M/R framework to not call commit for special
tasks like job setup/cleanup and task cleanup.
(Amareshwari Sriramadasu via yhemanth)
@@ -1703,7 +1704,7 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-670. Creates ant target for 10 mins patch test build.
(Jothi Padmanabhan via gkesavan)
- MAPREDUCE-1430. JobTracker should be able to renew delegation tokens
+ MAPREDUCE-1430. JobTracker should be able to renew delegation tokens
for the jobs(boryas)
HADOOP-6551, HDFS-986, MAPREDUCE-1503. Change API for tokens to throw
@@ -1711,7 +1712,7 @@ Release 0.20.202.0 - unreleased
HADOOP-6545. Changes the Key for the FileSystem to be UGI. (ddas)
- HADOOP-6572. Makes sure that SASL encryption and push to responder queue
+ HADOOP-6572. Makes sure that SASL encryption and push to responder queue
for the RPC response happens atomically. (Kan Zhang via ddas)
HDFS-965. Split the HDFS TestDelegationToken into two tests, of which
@@ -1727,8 +1728,8 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-1026. Fixes a bug in the backport. (ddas)
- HADOOP-6559. Makes the RPC client automatically re-login when the SASL
- connection setup fails. This is applicable to only keytab based logins.
+ HADOOP-6559. Makes the RPC client automatically re-login when the SASL
+ connection setup fails. This is applicable to only keytab based logins.
(ddas)
HADOOP-2141. Backport changes made in the original JIRA to aid
@@ -1737,23 +1738,23 @@ Release 0.20.202.0 - unreleased
HADOOP-6382. Import the mavenizable pom file structure and adjust
the build targets and bin scripts. (gkesvan via ltucker)
- MAPREDUCE-1425. archive throws OutOfMemoryError (mahadev)
+ MAPREDUCE-1425. archive throws OutOfMemoryError (mahadev)
MAPREDUCE-1399. The archive command shows a null error message. (nicholas)
- HADOOP-6552. Puts renewTGT=true and useTicketCache=true for the keytab
+ HADOOP-6552. Puts renewTGT=true and useTicketCache=true for the keytab
kerberos options. (ddas)
MAPREDUCE-1433. Adds delegation token for MapReduce (ddas)
HADOOP-4359. Fixes a bug in the earlier backport. (ddas)
- HADOOP-6547, HDFS-949, MAPREDUCE-1470. Move Delegation token into Common
- so that we can use it for MapReduce also. It is a combined patch for
+ HADOOP-6547, HDFS-949, MAPREDUCE-1470. Move Delegation token into Common
+ so that we can use it for MapReduce also. It is a combined patch for
common, hdfs and mr. (jitendra)
- HADOOP-6510,HDFS-935,MAPREDUCE-1464. Support for doAs to allow
- authenticated superuser to impersonate proxy users. It is a combined
+ HADOOP-6510,HDFS-935,MAPREDUCE-1464. Support for doAs to allow
+ authenticated superuser to impersonate proxy users. It is a combined
patch with compatible fixes in HDFS and MR. (jitendra)
MAPREDUCE-1435. Fixes the way symlinks are handled when cleaning up
@@ -1761,27 +1762,27 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-6419. Fixes a bug in the backported patch. (ddas)
- MAPREDUCE-1457. Fixes JobTracker to get the FileSystem object within
+ MAPREDUCE-1457. Fixes JobTracker to get the FileSystem object within
getStagingAreaDir within a privileged block. Fixes Child.java to use the
appropriate UGIs while getting the TaskUmbilicalProtocol proxy and while
executing the task. Contributed by Jakob Homan. (ddas)
MAPREDUCE-1440. Replace the long user name in MapReduce with the local
- name. (ddas)
+ name. (ddas)
- HADOOP-6419. Adds SASL based authentication to RPC. Also includes the
- MAPREDUCE-1335 and HDFS-933 patches. Contributed by Kan Zhang.
+ HADOOP-6419. Adds SASL based authentication to RPC. Also includes the
+ MAPREDUCE-1335 and HDFS-933 patches. Contributed by Kan Zhang.
(ddas)
-
+
HADOOP-6538. Sets hadoop.security.authentication to simple by default.
(ddas)
- HDFS-938. Replace calls to UGI.getUserName() with
+ HDFS-938. Replace calls to UGI.getUserName() with
UGI.getShortUserName()(boryas)
HADOOP-6544. fix ivy settings to include JSON jackson.codehause.org
libs for .20 (boryas)
-
+
HDFS-907. Add tests for getBlockLocations and totalLoad metrics. (rphulari)
HADOOP-6204. Implementing aspects development and fault injeciton
@@ -1792,9 +1793,9 @@ Release 0.20.202.0 - unreleased
the copying of files from the hdfs. (Devaraj Das vi boryas)
MAPREDUCE-1383. Automates fetching of delegation tokens in File*Formats
- Distributed Cache and Distcp. Also, provides a config
- mapreduce.job.hdfs-servers that the jobs can populate with a comma
- separated list of namenodes. The job client automatically fetches
+ Distributed Cache and Distcp. Also, provides a config
+ mapreduce.job.hdfs-servers that the jobs can populate with a comma
+ separated list of namenodes. The job client automatically fetches
delegation tokens from those namenodes.
HADOOP-6337. Update FilterInitializer class to be more visible
@@ -1802,13 +1803,13 @@ Release 0.20.202.0 - unreleased
HADOOP-6520. UGI should load tokens from the environment. (jitendra)
- HADOOP-6517, HADOOP-6518. Ability to add/get tokens from
+ HADOOP-6517, HADOOP-6518. Ability to add/get tokens from
UserGroupInformation & Kerberos login in UGI should honor KRB5CCNAME
(jitendra)
HADOOP-6299. Reimplement the UserGroupInformation to use the OS
specific and Kerberos JAAS login. (jhoman, ddas, oom)
-
+
HADOOP-6524. Contrib tests are failing Clover'ed build. (cos)
MAPREDUCE-842. Fixing a bug in the earlier version of the patch
@@ -1818,14 +1819,14 @@ Release 0.20.202.0 - unreleased
HDFS-919. Create test to validate the BlocksVerified metric (Gary Murry
via cos)
- MAPREDUCE-1186. Modified code in distributed cache to set
+ MAPREDUCE-1186. Modified code in distributed cache to set
permissions only on required set of localized paths.
(Amareshwari Sriramadasu via yhemanth)
HDFS-899. Delegation Token Implementation. (Jitendra Nath Pandey)
- MAPREDUCE-896. Enhance tasktracker to cleanup files that might have
- been created by user tasks with non-writable permissions.
+ MAPREDUCE-896. Enhance tasktracker to cleanup files that might have
+ been created by user tasks with non-writable permissions.
(Ravi Gummadi via yhemanth)
HADOOP-5879. Read compression level and strategy from Configuration for
@@ -1836,24 +1837,24 @@ Release 0.20.202.0 - unreleased
HADOOP-6382 Mavenize the build.xml targets and update the bin scripts
in preparation for publishing POM files (giri kesavan via ltucker)
- HDFS-737. Add full path name of the file to the block information and
- summary of total number of files, blocks, live and deadnodes to
+ HDFS-737. Add full path name of the file to the block information and
+ summary of total number of files, blocks, live and deadnodes to
metasave output. (Jitendra Nath Pandey via suresh)
HADOOP-6577. Add hidden configuration option "ipc.server.max.response.size"
- to change the default 1 MB, the maximum size when large IPC handler
+ to change the default 1 MB, the maximum size when large IPC handler
response buffer is reset. (suresh)
- HADOOP-6521. Fix backward compatiblity issue with umask when applications
- use deprecated param dfs.umask in configuration or use
+ HADOOP-6521. Fix backward compatiblity issue with umask when applications
+ use deprecated param dfs.umask in configuration or use
FsPermission.setUMask(). (suresh)
- HDFS-737. Add full path name of the file to the block information and
- summary of total number of files, blocks, live and deadnodes to
+ HDFS-737. Add full path name of the file to the block information and
+ summary of total number of files, blocks, live and deadnodes to
metasave output. (Jitendra Nath Pandey via suresh)
- HADOOP-6521. Fix backward compatiblity issue with umask when applications
- use deprecated param dfs.umask in configuration or use
+ HADOOP-6521. Fix backward compatiblity issue with umask when applications
+ use deprecated param dfs.umask in configuration or use
FsPermission.setUMask(). (suresh)
MAPREDUCE-433. Use more reliable counters in TestReduceFetch.
@@ -1862,18 +1863,18 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-744. Introduces the notion of a public distributed cache.
(ddas)
- MAPREDUCE-1140. Fix DistributedCache to not decrement reference counts
- for unreferenced files in error conditions.
+ MAPREDUCE-1140. Fix DistributedCache to not decrement reference counts
+ for unreferenced files in error conditions.
(Amareshwari Sriramadasu via yhemanth)
- MAPREDUCE-1284. Fix fts_open() call in task-controller that was failing
+ MAPREDUCE-1284. Fix fts_open() call in task-controller that was failing
LinuxTaskController unit tests. (Ravi Gummadi via yhemanth)
- MAPREDUCE-1098. Fixed the distributed-cache to not do i/o while
+ MAPREDUCE-1098. Fixed the distributed-cache to not do i/o while
holding a global lock.
(Amareshwari Sriramadasu via acmurthy)
- MAPREDUCE-1338. Introduces the notion of token cache using which
+ MAPREDUCE-1338. Introduces the notion of token cache using which
tokens and secrets can be sent by the Job client to the JobTracker.
(Boris Shkolnik)
@@ -1889,20 +1890,20 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-856. Setup secure permissions for distributed cache files.
(Vinod Kumar Vavilapalli via yhemanth)
- MAPREDUCE-871. Fix ownership of Job/Task local files to have correct
+ MAPREDUCE-871. Fix ownership of Job/Task local files to have correct
group ownership according to the egid of the tasktracker.
(Vinod Kumar Vavilapalli via yhemanth)
-
- MAPREDUCE-476. Extend DistributedCache to work locally (LocalJobRunner).
+
+ MAPREDUCE-476. Extend DistributedCache to work locally (LocalJobRunner).
(Philip Zeyliger via tomwhite)
- MAPREDUCE-711. Removed Distributed Cache from Common, to move it under
+ MAPREDUCE-711. Removed Distributed Cache from Common, to move it under
Map/Reduce. (Vinod Kumar Vavilapalli via yhemanth)
- MAPREDUCE-478. Allow map and reduce jvm parameters, environment
+ MAPREDUCE-478. Allow map and reduce jvm parameters, environment
variables and ulimit to be set separately. (acmurthy)
-
- MAPREDUCE-842. Setup secure permissions for localized job files,
+
+ MAPREDUCE-842. Setup secure permissions for localized job files,
intermediate outputs and log files on tasktrackers.
(Vinod Kumar Vavilapalli via yhemanth)
@@ -1915,16 +1916,16 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-181. Changes the job submission process to be secure.
(Devaraj Das)
- HADOOP-5737. Fixes a problem in the way the JobTracker used to talk to
- other daemons like the NameNode to get the job's files. Also adds APIs
- in the JobTracker to get the FileSystem objects as per the JobTracker's
+ HADOOP-5737. Fixes a problem in the way the JobTracker used to talk to
+ other daemons like the NameNode to get the job's files. Also adds APIs
+ in the JobTracker to get the FileSystem objects as per the JobTracker's
configuration. (Amar Kamat via ddas)
-
+
HADOOP-5771. Implements unit tests for LinuxTaskController.
(Sreekanth Ramakrishnan and Vinod Kumar Vavilapalli via yhemanth)
- HADOOP-4656, HDFS-685, MAPREDUCE-1083. Use the user-to-groups mapping
- service in the NameNode and JobTracker. Combined patch for these 3 jiras
+ HADOOP-4656, HDFS-685, MAPREDUCE-1083. Use the user-to-groups mapping
+ service in the NameNode and JobTracker. Combined patch for these 3 jiras
otherwise tests fail. (Jitendra Nath Pandey)
MAPREDUCE-1250. Refactor job token to use a common token interface.
@@ -1934,10 +1935,10 @@ Release 0.20.202.0 - unreleased
HADOOP-4268. Permission checking in fsck. (Jitendra Nath Pandey)
- HADOOP-6415. Adding a common token interface for both job token and
+ HADOOP-6415. Adding a common token interface for both job token and
delegation token. (Jitendra Nath Pandey)
- HADOOP-6367, HDFS-764. Moving Access Token implementation from Common to
+ HADOOP-6367, HDFS-764. Moving Access Token implementation from Common to
HDFS. These two jiras must be committed together otherwise build will
fail. (Jitendra Nath Pandey)
@@ -1950,32 +1951,32 @@ Release 0.20.202.0 - unreleased
HDFS-445. pread() fails when cached block locations are no longer valid.
(Jitendra Nath Pandey)
- HDFS-195. Need to handle access token expiration when re-establishing the
+ HDFS-195. Need to handle access token expiration when re-establishing the
pipeline for dfs write. (Jitendra Nath Pandey)
- HADOOP-6176. Adding a couple private methods to AccessTokenHandler
+ HADOOP-6176. Adding a couple private methods to AccessTokenHandler
for testing purposes. (Jitendra Nath Pandey)
HADOOP-5824. remove OP_READ_METADATA functionality from Datanode.
(Jitendra Nath Pandey)
- HADOOP-4359. Access Token: Support for data access authorization
+ HADOOP-4359. Access Token: Support for data access authorization
checking on DataNodes. (Jitendra Nath Pandey)
MAPREDUCE-1372. Fixed a ConcurrentModificationException in jobtracker.
(Arun C Murthy via yhemanth)
MAPREDUCE-1316. Fix jobs' retirement from the JobTracker to prevent memory
- leaks via stale references. (Amar Kamat via acmurthy)
+ leaks via stale references. (Amar Kamat via acmurthy)
- MAPREDUCE-1342. Fixed deadlock in global blacklisting of tasktrackers.
- (Amareshwari Sriramadasu via acmurthy)
+ MAPREDUCE-1342. Fixed deadlock in global blacklisting of tasktrackers.
+ (Amareshwari Sriramadasu via acmurthy)
HADOOP-6460. Reinitializes buffers used for serializing responses in ipc
server on exceeding maximum response size to free up Java heap. (suresh)
MAPREDUCE-1100. Truncate user logs to prevent TaskTrackers' disks from
- filling up. (Vinod Kumar Vavilapalli via acmurthy)
+ filling up. (Vinod Kumar Vavilapalli via acmurthy)
MAPREDUCE-1143. Fix running task counters to be updated correctly
when speculative attempts are running for a TIP.
@@ -1988,36 +1989,36 @@ Release 0.20.202.0 - unreleased
spurious logging in tasktracker logs for absent file paths.
(Ravi Gummadi via yhemanth)
- MAPREDUCE-676. Fix Hadoop Vaidya to ensure it works for map-only jobs.
- (Suhas Gogate via acmurthy)
+ MAPREDUCE-676. Fix Hadoop Vaidya to ensure it works for map-only jobs.
+ (Suhas Gogate via acmurthy)
HADOOP-5582. Fix Hadoop Vaidya to use new Counters in
- org.apache.hadoop.mapreduce package. (Suhas Gogate via acmurthy)
+ org.apache.hadoop.mapreduce package. (Suhas Gogate via acmurthy)
- HDFS-595. umask settings in configuration may now use octal or
+ HDFS-595. umask settings in configuration may now use octal or
symbolic instead of decimal. Update HDFS tests as such. (jghoman)
MAPREDUCE-1068. Added a verbose error message when user specifies an
- incorrect -file parameter. (Amareshwari Sriramadasu via acmurthy)
+ incorrect -file parameter. (Amareshwari Sriramadasu via acmurthy)
MAPREDUCE-1171. Allow the read-error notification in shuffle to be
- configurable. (Amareshwari Sriramadasu via acmurthy)
+ configurable. (Amareshwari Sriramadasu via acmurthy)
MAPREDUCE-353. Allow shuffle read and connection timeouts to be
- configurable. (Amareshwari Sriramadasu via acmurthy)
+ configurable. (Amareshwari Sriramadasu via acmurthy)
- HDFS-781. Namenode metrics PendingDeletionBlocks is not decremented.
+ HDFS-781. Namenode metrics PendingDeletionBlocks is not decremented.
(suresh)
-
- MAPREDUCE-1185. Redirect running job url to history url if job is already
+
+ MAPREDUCE-1185. Redirect running job url to history url if job is already
retired. (Amareshwari Sriramadasu and Sharad Agarwal via sharad)
- MAPREDUCE-754. Fix NPE in expiry thread when a TT is lost. (Amar Kamat
+ MAPREDUCE-754. Fix NPE in expiry thread when a TT is lost. (Amar Kamat
via sharad)
MAPREDUCE-896. Modify permissions for local files on tasktracker before
deletion so they can be deleted cleanly. (Ravi Gummadi via yhemanth)
-
+
HADOOP-5771. Implements unit tests for LinuxTaskController.
(Sreekanth Ramakrishnan and Vinod Kumar Vavilapalli via yhemanth)
@@ -2041,7 +2042,7 @@ Release 0.20.202.0 - unreleased
when the history viewer is accessed concurrently.
(Amar Kamat via ddas)
- MAPREDUCE-1140. Fix DistributedCache to not decrement reference counts for
+ MAPREDUCE-1140. Fix DistributedCache to not decrement reference counts for
unreferenced files in error conditions.
(Amareshwari Sriramadasu via yhemanth)
@@ -2056,10 +2057,10 @@ Release 0.20.202.0 - unreleased
This fix is a port from hdfs-0.22 to common-0.20 by Boris Shkolnik.
Contributed by Erik Steffl
- HDFS-625. Fix NullPointerException thrown from ListPathServlet.
+ HDFS-625. Fix NullPointerException thrown from ListPathServlet.
Contributed by Suresh Srinivas.
- HADOOP-6343. Log unexpected throwable object caught in RPC.
+ HADOOP-6343. Log unexpected throwable object caught in RPC.
Contributed by Jitendra Nath Pandey
MAPREDUCE-1186. Fixed DistributedCache to do a recursive chmod on just the
@@ -2114,7 +2115,7 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-947. Added commitJob and abortJob apis to OutputCommitter.
Enhanced FileOutputCommitter to create a _SUCCESS file for successful
- jobs. (Amar Kamat & Jothi Padmanabhan via acmurthy)
+ jobs. (Amar Kamat & Jothi Padmanabhan via acmurthy)
MAPREDUCE-1105. Remove max limit configuration in capacity scheduler in
favor of max capacity percentage thus allowing the limit to go over
@@ -2123,7 +2124,7 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-1086. Setup Hadoop logging environment for tasks to point to
task related parameters. (Ravi Gummadi via yhemanth)
- MAPREDUCE-739. Allow relative paths to be created inside archives.
+ MAPREDUCE-739. Allow relative paths to be created inside archives.
(mahadev)
HADOOP-6097. Multiple bugs w/ Hadoop archives (mahadev)
@@ -2137,7 +2138,7 @@ Release 0.20.202.0 - unreleased
HDFS-686. NullPointerException is thrown while merging edit log and
image. (hairong)
- HDFS-709. Fix TestDFSShell failure due to rename bug introduced by
+ HDFS-709. Fix TestDFSShell failure due to rename bug introduced by
HDFS-677. (suresh)
HDFS-677. Rename failure when both source and destination quota exceeds
@@ -2150,18 +2151,18 @@ Release 0.20.202.0 - unreleased
MAPREDUCE-732. Removed spurious log statements in the node
blacklisting logic. (Sreekanth Ramakrishnan via yhemanth)
- MAPREDUCE-144. Includes dump of the process tree in task diagnostics when
+ MAPREDUCE-144. Includes dump of the process tree in task diagnostics when
a task is killed due to exceeding memory limits.
(Vinod Kumar Vavilapalli via yhemanth)
- MAPREDUCE-979. Fixed JobConf APIs related to memory parameters to
- return values of new configuration variables when deprecated
+ MAPREDUCE-979. Fixed JobConf APIs related to memory parameters to
+ return values of new configuration variables when deprecated
variables are disabled. (Sreekanth Ramakrishnan via yhemanth)
MAPREDUCE-277. Makes job history counters available on the job history
viewers. (Jothi Padmanabhan via ddas)
- HADOOP-5625. Add operation duration to clienttrace. (Lei Xu
+ HADOOP-5625. Add operation duration to clienttrace. (Lei Xu
via cdouglas)
HADOOP-5222. Add offset to datanode clienttrace. (Lei Xu via cdouglas)
@@ -2174,7 +2175,7 @@ Release 0.20.202.0 - unreleased
HADOOP-6304. Use java.io.File.set{Readable|Writable|Executable} where
possible in RawLocalFileSystem. Contributed by Arun C. Murthy.
-
+
MAPREDUCE-270. Fix the tasktracker to optionally send an out-of-band
heartbeat on task-completion for better job-latency. Contributed by
Arun C. Murthy
@@ -2185,66 +2186,66 @@ Release 0.20.202.0 - unreleased
per-heartbeat. Contributed by Rahuk K Singh.
MAPREDUCE-1028. Fixed number of slots occupied by cleanup tasks to one
- irrespective of slot size for the job. Contributed by Ravi Gummadi.
+ irrespective of slot size for the job. Contributed by Ravi Gummadi.
MAPREDUCE-964. Fixed start and finish times of TaskStatus to be
consistent, thereby fixing inconsistencies in metering tasks.
Contributed by Sreekanth Ramakrishnan.
- HADOOP-5976. Add a new command, classpath, to the hadoop
+ HADOOP-5976. Add a new command, classpath, to the hadoop
script. Contributed by Owen O'Malley and Gary Murry
- HADOOP-5784. Makes the number of heartbeats that should arrive
- a second at the JobTracker configurable. Contributed by
+ HADOOP-5784. Makes the number of heartbeats that should arrive
+ a second at the JobTracker configurable. Contributed by
Amareshwari Sriramadasu.
- MAPREDUCE-945. Modifies MRBench and TestMapRed to use
- ToolRunner so that options such as queue name can be
+ MAPREDUCE-945. Modifies MRBench and TestMapRed to use
+ ToolRunner so that options such as queue name can be
passed via command line. Contributed by Sreekanth Ramakrishnan.
HADOOP:5420 Correct bug in earlier implementation
by Arun C. Murthy
- HADOOP-5363 Add support for proxying connections to multiple
- clusters with different versions to hdfsproxy. Contributed
+ HADOOP-5363 Add support for proxying connections to multiple
+ clusters with different versions to hdfsproxy. Contributed
by Zhiyong Zhang
- HADOOP-5780. Improve per block message prited by -metaSave
+ HADOOP-5780. Improve per block message prited by -metaSave
in HDFS. (Raghu Angadi)
- HADOOP-6227. Fix Configuration to allow final parameters to be set
- to null and prevent them from being overridden. Contributed by
+ HADOOP-6227. Fix Configuration to allow final parameters to be set
+ to null and prevent them from being overridden. Contributed by
Amareshwari Sriramadasu.
MAPREDUCE-430 Added patch supplied by Amar Kamat to allow roll forward
on branch to includ externally committed patch.
- MAPREDUCE-768. Provide an option to dump jobtracker configuration in
+ MAPREDUCE-768. Provide an option to dump jobtracker configuration in
JSON format to standard output. Contributed by V.V.Chaitanya
MAPREDUCE-834 Correct an issue created by merging this issue with
patch attached to external Jira.
- HADOOP-6184 Provide an API to dump Configuration in a JSON format.
+ HADOOP-6184 Provide an API to dump Configuration in a JSON format.
Contributed by V.V.Chaitanya Krishna.
- MAPREDUCE-745 Patch added for this issue to allow branch-0.20 to
+ MAPREDUCE-745 Patch added for this issue to allow branch-0.20 to
merge cleanly.
- MAPREDUCE:478 Allow map and reduce jvm parameters, environment
+ MAPREDUCE:478 Allow map and reduce jvm parameters, environment
variables and ulimit to be set separately.
- MAPREDUCE:682 Removes reservations on tasktrackers which are blacklisted.
+ MAPREDUCE:682 Removes reservations on tasktrackers which are blacklisted.
Contributed by Sreekanth Ramakrishnan.
- HADOOP:5420 Support killing of process groups in LinuxTaskController
+ HADOOP:5420 Support killing of process groups in LinuxTaskController
binary
- HADOOP-5488 Removes the pidfile management for the Task JVM from the
- framework and instead passes the PID back and forth between the
+ HADOOP-5488 Removes the pidfile management for the Task JVM from the
+ framework and instead passes the PID back and forth between the
TaskTracker and the Task processes. Contributed by Ravi Gummadi.
- MAPREDUCE:467 Provide ability to collect statistics about total tasks and
+ MAPREDUCE:467 Provide ability to collect statistics about total tasks and
succeeded tasks in different time windows.
MAPREDUCE-817. Add a cache for retired jobs with minimal job
@@ -2257,8 +2258,8 @@ Release 0.20.202.0 - unreleased
happens. The bug was that even if commit failed, the task would be
declared as successful. Contributed by Amareshwari Sriramadasu.
- MAPREDUCE-809 Fix job-summary logs to correctly record final status of
- FAILED and KILLED jobs.
+ MAPREDUCE-809 Fix job-summary logs to correctly record final status of
+ FAILED and KILLED jobs.
MAPREDUCE-740 Log a job-summary at the end of a job, while
allowing it to be configured to use a custom appender if desired.
@@ -2340,7 +2341,7 @@ Release 0.20.202.0 - unreleased
HADOOP-2838 Now the users can set environment variables using
mapred.child.env. They can do the following X=Y : set X to Y X=$X:Y
: Append Y to X (which should be taken from the tasktracker)
-
+
HADOOP-5818. Revert the renaming from FSNamesystem.checkSuperuserPrivilege
to checkAccess by HADOOP-5643. (Amar Kamat via szetszwo)
@@ -2348,7 +2349,7 @@ Release 0.20.202.0 - unreleased
then it should be refreshed upon recovery so that the excluded hosts are
lost and the maps are re-executed. (Amar Kamat via ddas)
- HADOOP-5643. HADOOP-5643. Adds a way to decommission TaskTrackers
+ HADOOP-5643. HADOOP-5643. Adds a way to decommission TaskTrackers
while the JobTracker is running. (Amar Kamat via ddas)
HADOOP-5419. Provide a facility to query the Queue ACLs for the
@@ -2358,7 +2359,7 @@ Release 0.20.202.0 - unreleased
JobTracker metrics. (Sreekanth Ramakrishnan via cdouglas)
HADOOP-5738. Split "waiting_tasks" JobTracker metric into waiting maps and
- waiting reduces. (Sreekanth Ramakrishnan via cdouglas)
+ waiting reduces. (Sreekanth Ramakrishnan via cdouglas)
HADOOP-4842. Streaming now allows specifiying a command for the combiner.
(Amareshwari Sriramadasu via ddas)
@@ -2389,7 +2390,7 @@ Release 0.20.3 - Unreleased
BUG FIXES
- HDFS-955. New implementation of saveNamespace() to avoid loss of edits
+ HDFS-955. New implementation of saveNamespace() to avoid loss of edits
when name-node fails during saving. (shv)
Release 0.20.2 - 2010-2-16
@@ -2437,9 +2438,9 @@ Release 0.20.2 - 2010-2-16
MAPREDUCE-1070. Prevent a deadlock in the fair scheduler servlet.
(Todd Lipcon via cdouglas)
- HADOOP-5623. Ensure streaming status messages aren't overwritten. (Rick
+ HADOOP-5623. Ensure streaming status messages aren't overwritten. (Rick
Cox & Ravi Gummadi via tomwhite)
-
+
MAPREDUCE-1163. Remove unused, hard-coded paths from libhdfs. (Allen
Wittenauer via cdouglas)
@@ -2477,7 +2478,7 @@ Release 0.20.1 - 2009-09-01
like memory based scheduling, job initialization and removal of pre-emption.
(Sreekanth Ramakrishnan via yhemanth)
- HADOOP-4674. Fix fs help messages for -test, -text, -tail, -stat
+ HADOOP-4674. Fix fs help messages for -test, -text, -tail, -stat
and -touchz options. (Ravi Phulari via szetszwo)
HADOOP-4372. Improves the way history filenames are obtained and manipulated.
@@ -2495,10 +2496,10 @@ Release 0.20.1 - 2009-09-01
BUG FIXES
- HADOOP-5691. Makes org.apache.hadoop.mapreduce.Reducer concrete class
+ HADOOP-5691. Makes org.apache.hadoop.mapreduce.Reducer concrete class
instead of abstract. (Amareshwari Sriramadasu via sharad)
- HADOOP-5646. Fixes a problem in TestQueueCapacities.
+ HADOOP-5646. Fixes a problem in TestQueueCapacities.
(Vinod Kumar Vavilapalli via ddas)
HADOOP-5655. TestMRServerPorts fails on java.net.BindException. (Devaraj
@@ -2510,7 +2511,7 @@ Release 0.20.1 - 2009-09-01
HADOOP-5688. Fix HftpFileSystem checksum path construction. (Tsz Wo
(Nicholas) Sze via cdouglas)
- HADOOP-5213. Fix Null pointer exception caused when bzip2compression
+ HADOOP-5213. Fix Null pointer exception caused when bzip2compression
was used and user closed a output stream without writing any data.
(Zheng Shao via dhruba)
@@ -2529,7 +2530,7 @@ Release 0.20.1 - 2009-09-01
is of an unknown size. (Vinod Kumar Vavilapalli via ddas)
HADOOP-5636. Prevents a job from going to RUNNING state after it has been
- KILLED (this used to happen when the SetupTask would come back with a
+ KILLED (this used to happen when the SetupTask would come back with a
success after the job has been killed). (Amar Kamat via ddas)
HADOOP-5641. Fix a NullPointerException in capacity scheduler's memory
@@ -2545,11 +2546,11 @@ Release 0.20.1 - 2009-09-01
HADOOP-5210. Solves a problem in the progress report of the reduce task.
(Ravi Gummadi via ddas)
- HADOOP-5850. Fixes a problem to do with not being able to jobs with
+ HADOOP-5850. Fixes a problem to do with not being able to jobs with
0 maps/reduces. (Vinod K V via ddas)
-
+
HADOOP-5728. Fixed FSEditLog.printStatistics IndexOutOfBoundsException.
- (Wang Xu via johan)
+ (Wang Xu via johan)
HADOOP-4626. Correct the API links in hdfs forrest doc so that they
point to the same version of hadoop. (szetszwo)
@@ -2569,19 +2570,19 @@ Release 0.20.1 - 2009-09-01
HADOOP-5648. Fixes a build issue in not being able to generate gridmix.jar
in hadoop binary tarball. (Giridharan Kesavan via gkesavan)
- HADOOP-5908. Fixes a problem to do with ArithmeticException in the
+ HADOOP-5908. Fixes a problem to do with ArithmeticException in the
JobTracker when there are jobs with 0 maps. (Amar Kamat via ddas)
[... 3729 lines stripped ...]