You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by na...@apache.org on 2017/09/03 23:54:53 UTC
[13/77] [abbrv] [partial] hadoop git commit: HADOOP-14364. refresh
changelog/release notes with newer Apache Yetus build
http://git-wip-us.apache.org/repos/asf/hadoop/blob/19041008/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.2-alpha/RELEASENOTES.2.0.2-alpha.md
----------------------------------------------------------------------
diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.2-alpha/RELEASENOTES.2.0.2-alpha.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.2-alpha/RELEASENOTES.2.0.2-alpha.md
index 7073417..77aa1a0 100644
--- a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.2-alpha/RELEASENOTES.2.0.2-alpha.md
+++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.2-alpha/RELEASENOTES.2.0.2-alpha.md
@@ -23,65 +23,63 @@ These release notes cover new developer and user-facing incompatibilities, impor
---
-* [HADOOP-8794](https://issues.apache.org/jira/browse/HADOOP-8794) | *Major* | **Modifiy bin/hadoop to point to HADOOP\_YARN\_HOME**
+* [HADOOP-7703](https://issues.apache.org/jira/browse/HADOOP-7703) | *Major* | **WebAppContext should also be stopped and cleared**
-**WARNING: No release note provided for this incompatible change.**
+Improved excpetion handling of shutting down web server. (Devaraj K via Eric Yang)
---
-* [HADOOP-8710](https://issues.apache.org/jira/browse/HADOOP-8710) | *Major* | **Remove ability for users to easily run the trash emptier**
+* [MAPREDUCE-3348](https://issues.apache.org/jira/browse/MAPREDUCE-3348) | *Major* | **mapred job -status fails to give info even if the job is present in History**
-The trash emptier may no longer be run using "hadoop org.apache.hadoop.fs.Trash". The trash emptier runs on the NameNode (if configured). Old trash checkpoints may be deleted using "hadoop fs -expunge".
+Fixed a bug in MR client to redirect to JobHistoryServer correctly when RM forgets the app.
---
-* [HADOOP-8703](https://issues.apache.org/jira/browse/HADOOP-8703) | *Major* | **distcpV2: turn CRC checking off for 0 byte size**
+* [MAPREDUCE-4072](https://issues.apache.org/jira/browse/MAPREDUCE-4072) | *Major* | **User set java.library.path seems to overwrite default creating problems native lib loading**
-distcp skips CRC on 0 byte files.
+-Djava.library.path in mapred.child.java.opts can cause issues with native libraries. LD\_LIBRARY\_PATH through mapred.child.env should be used instead.
---
-* [HADOOP-8689](https://issues.apache.org/jira/browse/HADOOP-8689) | *Major* | **Make trash a server side configuration option**
+* [HDFS-3110](https://issues.apache.org/jira/browse/HDFS-3110) | *Major* | **libhdfs implementation of direct read API**
-If fs.trash.interval is configured on the server then the client's value for this configuration is ignored.
+libhdfs is enhanced to read directly into user-supplied buffers when possible, reducing the number of memory copies.
---
-* [HADOOP-8551](https://issues.apache.org/jira/browse/HADOOP-8551) | *Major* | **fs -mkdir creates parent directories without the -p option**
+* [MAPREDUCE-4017](https://issues.apache.org/jira/browse/MAPREDUCE-4017) | *Trivial* | **Add jobname to jobsummary log**
-FsShell's "mkdir" no longer implicitly creates all non-existent parent directories. The command adopts the posix compliant behavior of requiring the "-p" flag to auto-create parent directories.
+The Job Summary log may contain commas in values that are escaped by a '\\' character. This was true before, but is more likely to be exposed now.
---
-* [HADOOP-8533](https://issues.apache.org/jira/browse/HADOOP-8533) | *Major* | **Remove Parallel Call in IPC**
-
-Merged the change to branch-2
-
-
----
+* [MAPREDUCE-3812](https://issues.apache.org/jira/browse/MAPREDUCE-3812) | *Major* | **Lower default allocation sizes, fix allocation configurations and document them**
-* [HADOOP-8458](https://issues.apache.org/jira/browse/HADOOP-8458) | *Major* | **Add management hook to AuthenticationHandler to enable delegation token operations support**
+Removes two sets of previously available config properties:
-**WARNING: No release note provided for this incompatible change.**
+1. ( yarn.scheduler.fifo.minimum-allocation-mb and yarn.scheduler.fifo.maximum-allocation-mb ) and,
+2. ( yarn.scheduler.capacity.minimum-allocation-mb and yarn.scheduler.capacity.maximum-allocation-mb )
+In favor of two new, generically named properties:
----
+1. yarn.scheduler.minimum-allocation-mb - This acts as the floor value of memory resource requests for containers.
+2. yarn.scheduler.maximum-allocation-mb - This acts as the ceiling value of memory resource requests for containers.
-* [HADOOP-8388](https://issues.apache.org/jira/browse/HADOOP-8388) | *Minor* | **Remove unused BlockLocation serialization**
+Both these properties need to be set at the ResourceManager (RM) to take effect, as the RM is where the scheduler resides.
-**WARNING: No release note provided for this incompatible change.**
+Also changes the default minimum and maximums to 128 MB and 10 GB respectively.
---
-* [HADOOP-8368](https://issues.apache.org/jira/browse/HADOOP-8368) | *Minor* | **Use CMake rather than autotools to build native code**
+* [HDFS-3318](https://issues.apache.org/jira/browse/HDFS-3318) | *Blocker* | **Hftp hangs on transfers \>2GB**
-**WARNING: No release note provided for this incompatible change.**
+**WARNING: No release note provided for this change.**
---
@@ -93,30 +91,32 @@ Resolve sporadic distcp issue due to having two DistCp classes (v1 & v2) in the
---
-* [HADOOP-7703](https://issues.apache.org/jira/browse/HADOOP-7703) | *Major* | **WebAppContext should also be stopped and cleared**
+* [HADOOP-8388](https://issues.apache.org/jira/browse/HADOOP-8388) | *Minor* | **Remove unused BlockLocation serialization**
-Improved excpetion handling of shutting down web server. (Devaraj K via Eric Yang)
+**WARNING: No release note provided for this change.**
---
-* [HDFS-3755](https://issues.apache.org/jira/browse/HDFS-3755) | *Major* | **Creating an already-open-for-write file with overwrite=true fails**
+* [MAPREDUCE-3543](https://issues.apache.org/jira/browse/MAPREDUCE-3543) | *Critical* | **Mavenize Gridmix.**
-This is an incompatible change: Before this change, if a file is already open for write by one client, and another client calls fs.create() with overwrite=true, an AlreadyBeingCreatedException is thrown. After this change, the file will be deleted and the new file will be created successfully.
+Note that to apply this you should first run the script - ./MAPREDUCE-3543v3.sh svn, then apply the patch.
+
+If this is merged to more then trunk, the version inside of hadoop-tools/hadoop-gridmix/pom.xml will need to be udpated accordingly.
---
-* [HDFS-3697](https://issues.apache.org/jira/browse/HDFS-3697) | *Minor* | **Enable fadvise readahead by default**
+* [MAPREDUCE-3873](https://issues.apache.org/jira/browse/MAPREDUCE-3873) | *Minor* | **Nodemanager is not getting decommisioned if the absolute ip is given in exclude file.**
-The datanode now performs 4MB readahead by default when reading data from its disks, if the native libraries are present. This has been shown to improve performance in many workloads. The feature may be disabled by setting dfs.datanode.readahead.bytes to "0".
+Fixed NodeManagers' decommissioning at RM to accept IP addresses also.
---
-* [HDFS-3675](https://issues.apache.org/jira/browse/HDFS-3675) | *Minor* | **libhdfs: follow documented return codes**
+* [HADOOP-8368](https://issues.apache.org/jira/browse/HADOOP-8368) | *Minor* | **Use CMake rather than autotools to build native code**
-**WARNING: No release note provided for this incompatible change.**
+**WARNING: No release note provided for this change.**
---
@@ -128,6 +128,13 @@ getBlockLocations(), and hence open() for read, will now throw SafeModeException
---
+* [HADOOP-8458](https://issues.apache.org/jira/browse/HADOOP-8458) | *Major* | **Add management hook to AuthenticationHandler to enable delegation token operations support**
+
+**WARNING: No release note provided for this change.**
+
+
+---
+
* [HDFS-3518](https://issues.apache.org/jira/browse/HDFS-3518) | *Major* | **Provide API to check HDFS operational state**
Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy.
@@ -135,6 +142,13 @@ Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is
---
+* [MAPREDUCE-4311](https://issues.apache.org/jira/browse/MAPREDUCE-4311) | *Major* | **Capacity scheduler.xml does not accept decimal values for capacity and maximum-capacity settings**
+
+**WARNING: No release note provided for this change.**
+
+
+---
+
* [HDFS-3475](https://issues.apache.org/jira/browse/HDFS-3475) | *Trivial* | **Make the replication and invalidation rates configurable**
This change adds two new configuration parameters.
@@ -153,121 +167,107 @@ HDFS no longer silently ignores missing or unreadable host files specified by df
---
-* [HDFS-3318](https://issues.apache.org/jira/browse/HDFS-3318) | *Blocker* | **Hftp hangs on transfers \>2GB**
+* [HADOOP-8533](https://issues.apache.org/jira/browse/HADOOP-8533) | *Major* | **Remove Parallel Call in IPC**
-**WARNING: No release note provided for this incompatible change.**
+Merged the change to branch-2
---
-* [HDFS-3110](https://issues.apache.org/jira/browse/HDFS-3110) | *Major* | **libhdfs implementation of direct read API**
+* [MAPREDUCE-3940](https://issues.apache.org/jira/browse/MAPREDUCE-3940) | *Major* | **ContainerTokens should have an expiry interval**
-libhdfs is enhanced to read directly into user-supplied buffers when possible, reducing the number of memory copies.
+ContainerTokens now have an expiry interval so that stale tokens cannot be used for launching containers.
---
-* [HDFS-2793](https://issues.apache.org/jira/browse/HDFS-2793) | *Major* | **Add an admin command to trigger an edit log roll**
+* [HDFS-3675](https://issues.apache.org/jira/browse/HDFS-3675) | *Minor* | **libhdfs: follow documented return codes**
-Introduced a new command, "hdfs dfsadmin -rollEdits" which requests that the active NameNode roll its edit log. This can be useful for administrators manually backing up log segments.
+**WARNING: No release note provided for this change.**
---
-* [HDFS-2727](https://issues.apache.org/jira/browse/HDFS-2727) | *Minor* | **libhdfs should get the default block size from the server**
+* [HDFS-2617](https://issues.apache.org/jira/browse/HDFS-2617) | *Major* | **Replaced Kerberized SSL for image transfer and fsck with SPNEGO-based solution**
-libhdfs now uses the server block size configuration rather than the deprecated dfs.block.size client configuration.
+Due to the requirement that KSSL use weak encryption types for Kerberos tickets, HTTP authentication to the NameNode will now use SPNEGO by default. This will require users of previous branch-1 releases with security enabled to modify their configurations and create new Kerberos principals in order to use SPNEGO. The old behavior of using KSSL can optionally be enabled by setting the configuration option "hadoop.security.use-weak-http-crypto" to "true".
---
-* [HDFS-2686](https://issues.apache.org/jira/browse/HDFS-2686) | *Major* | **Remove DistributedUpgrade related code**
+* [HADOOP-8551](https://issues.apache.org/jira/browse/HADOOP-8551) | *Major* | **fs -mkdir creates parent directories without the -p option**
-This jira removes functionality that has not been used/applicable since release 0.17. The incompatibility introduced by this change will not affect any HDFS users.
+FsShell's "mkdir" no longer implicitly creates all non-existent parent directories. The command adopts the posix compliant behavior of requiring the "-p" flag to auto-create parent directories.
---
-* [HDFS-2617](https://issues.apache.org/jira/browse/HDFS-2617) | *Major* | **Replaced Kerberized SSL for image transfer and fsck with SPNEGO-based solution**
+* [HDFS-3697](https://issues.apache.org/jira/browse/HDFS-3697) | *Minor* | **Enable fadvise readahead by default**
-Due to the requirement that KSSL use weak encryption types for Kerberos tickets, HTTP authentication to the NameNode will now use SPNEGO by default. This will require users of previous branch-1 releases with security enabled to modify their configurations and create new Kerberos principals in order to use SPNEGO. The old behavior of using KSSL can optionally be enabled by setting the configuration option "hadoop.security.use-weak-http-crypto" to "true".
+The datanode now performs 4MB readahead by default when reading data from its disks, if the native libraries are present. This has been shown to improve performance in many workloads. The feature may be disabled by setting dfs.datanode.readahead.bytes to "0".
---
-* [MAPREDUCE-4629](https://issues.apache.org/jira/browse/MAPREDUCE-4629) | *Major* | **Remove JobHistory.DEBUG\_MODE**
+* [HDFS-3755](https://issues.apache.org/jira/browse/HDFS-3755) | *Major* | **Creating an already-open-for-write file with overwrite=true fails**
-**WARNING: No release note provided for this incompatible change.**
+This is an incompatible change: Before this change, if a file is already open for write by one client, and another client calls fs.create() with overwrite=true, an AlreadyBeingCreatedException is thrown. After this change, the file will be deleted and the new file will be created successfully.
---
-* [MAPREDUCE-4311](https://issues.apache.org/jira/browse/MAPREDUCE-4311) | *Major* | **Capacity scheduler.xml does not accept decimal values for capacity and maximum-capacity settings**
+* [HADOOP-8703](https://issues.apache.org/jira/browse/HADOOP-8703) | *Major* | **distcpV2: turn CRC checking off for 0 byte size**
-**WARNING: No release note provided for this incompatible change.**
+distcp skips CRC on 0 byte files.
---
-* [MAPREDUCE-4072](https://issues.apache.org/jira/browse/MAPREDUCE-4072) | *Major* | **User set java.library.path seems to overwrite default creating problems native lib loading**
+* [HADOOP-8689](https://issues.apache.org/jira/browse/HADOOP-8689) | *Major* | **Make trash a server side configuration option**
--Djava.library.path in mapred.child.java.opts can cause issues with native libraries. LD\_LIBRARY\_PATH through mapred.child.env should be used instead.
+If fs.trash.interval is configured on the server then the client's value for this configuration is ignored.
---
-* [MAPREDUCE-4017](https://issues.apache.org/jira/browse/MAPREDUCE-4017) | *Trivial* | **Add jobname to jobsummary log**
+* [HADOOP-8710](https://issues.apache.org/jira/browse/HADOOP-8710) | *Major* | **Remove ability for users to easily run the trash emptier**
-The Job Summary log may contain commas in values that are escaped by a '\' character. This was true before, but is more likely to be exposed now.
+The trash emptier may no longer be run using "hadoop org.apache.hadoop.fs.Trash". The trash emptier runs on the NameNode (if configured). Old trash checkpoints may be deleted using "hadoop fs -expunge".
---
-* [MAPREDUCE-3940](https://issues.apache.org/jira/browse/MAPREDUCE-3940) | *Major* | **ContainerTokens should have an expiry interval**
+* [HDFS-2727](https://issues.apache.org/jira/browse/HDFS-2727) | *Minor* | **libhdfs should get the default block size from the server**
-ContainerTokens now have an expiry interval so that stale tokens cannot be used for launching containers.
+libhdfs now uses the server block size configuration rather than the deprecated dfs.block.size client configuration.
---
-* [MAPREDUCE-3873](https://issues.apache.org/jira/browse/MAPREDUCE-3873) | *Minor* | **Nodemanager is not getting decommisioned if the absolute ip is given in exclude file.**
+* [HDFS-2686](https://issues.apache.org/jira/browse/HDFS-2686) | *Major* | **Remove DistributedUpgrade related code**
-Fixed NodeManagers' decommissioning at RM to accept IP addresses also.
+This jira removes functionality that has not been used/applicable since release 0.17. The incompatibility introduced by this change will not affect any HDFS users.
---
-* [MAPREDUCE-3812](https://issues.apache.org/jira/browse/MAPREDUCE-3812) | *Major* | **Lower default allocation sizes, fix allocation configurations and document them**
-
-Removes two sets of previously available config properties:
-
-1. ( yarn.scheduler.fifo.minimum-allocation-mb and yarn.scheduler.fifo.maximum-allocation-mb ) and,
-2. ( yarn.scheduler.capacity.minimum-allocation-mb and yarn.scheduler.capacity.maximum-allocation-mb )
-
-In favor of two new, generically named properties:
-
-1. yarn.scheduler.minimum-allocation-mb - This acts as the floor value of memory resource requests for containers.
-2. yarn.scheduler.maximum-allocation-mb - This acts as the ceiling value of memory resource requests for containers.
-
-Both these properties need to be set at the ResourceManager (RM) to take effect, as the RM is where the scheduler resides.
+* [HDFS-2793](https://issues.apache.org/jira/browse/HDFS-2793) | *Major* | **Add an admin command to trigger an edit log roll**
-Also changes the default minimum and maximums to 128 MB and 10 GB respectively.
+Introduced a new command, "hdfs dfsadmin -rollEdits" which requests that the active NameNode roll its edit log. This can be useful for administrators manually backing up log segments.
---
-* [MAPREDUCE-3543](https://issues.apache.org/jira/browse/MAPREDUCE-3543) | *Critical* | **Mavenize Gridmix.**
-
-Note that to apply this you should first run the script - ./MAPREDUCE-3543v3.sh svn, then apply the patch.
+* [MAPREDUCE-4629](https://issues.apache.org/jira/browse/MAPREDUCE-4629) | *Major* | **Remove JobHistory.DEBUG\_MODE**
-If this is merged to more then trunk, the version inside of hadoop-tools/hadoop-gridmix/pom.xml will need to be udpated accordingly.
+**WARNING: No release note provided for this change.**
---
-* [MAPREDUCE-3348](https://issues.apache.org/jira/browse/MAPREDUCE-3348) | *Major* | **mapred job -status fails to give info even if the job is present in History**
+* [HADOOP-8794](https://issues.apache.org/jira/browse/HADOOP-8794) | *Major* | **Modifiy bin/hadoop to point to HADOOP\_YARN\_HOME**
-Fixed a bug in MR client to redirect to JobHistoryServer correctly when RM forgets the app.
+**WARNING: No release note provided for this change.**
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org