You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-commits@hadoop.apache.org by vi...@apache.org on 2010/04/29 08:39:50 UTC
svn commit: r939217 - in /hadoop/mapreduce/trunk: CHANGES.txt
src/docs/src/documentation/content/xdocs/mapred_tutorial.xml
Author: vinodkv
Date: Thu Apr 29 06:39:50 2010
New Revision: 939217
URL: http://svn.apache.org/viewvc?rev=939217&view=rev
Log:
MAPREDUCE-1417. Forrest documentation should be updated to reflect the changes in MAPREDUCE-744. Contributed by Ravi Gummadi.
Modified:
hadoop/mapreduce/trunk/CHANGES.txt
hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml
Modified: hadoop/mapreduce/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/trunk/CHANGES.txt?rev=939217&r1=939216&r2=939217&view=diff
==============================================================================
--- hadoop/mapreduce/trunk/CHANGES.txt (original)
+++ hadoop/mapreduce/trunk/CHANGES.txt Thu Apr 29 06:39:50 2010
@@ -275,6 +275,9 @@ Trunk (unreleased changes)
MAPREDUCE-1625. Improve grouping of packages in Javadoc. (tomwhite)
+ MAPREDUCE-1417. Forrest documentation should be updated to reflect
+ the changes in MAPREDUCE-744. (Ravi Gummadi via vinodkv)
+
OPTIMIZATIONS
MAPREDUCE-270. Fix the tasktracker to optionally send an out-of-band
Modified: hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml?rev=939217&r1=939216&r2=939217&view=diff
==============================================================================
--- hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml (original)
+++ hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml Thu Apr 29 06:39:50 2010
@@ -2022,6 +2022,37 @@
cached files that are symlinked into the working directory of the
task can be used to distribute native libraries and load them.</p>
+ <section>
+ <title>Private and Public DistributedCache Files</title>
+ <p>DistributedCache files can be private or public, that
+ determines how they can be shared on the slave nodes.</p>
+ <ul>
+ <li>"Private" DistributedCache files are cached in a local
+ directory private to the user whose jobs need these
+ files. These files are shared by all
+ tasks and jobs of the specific user only and cannot be accessed by
+ jobs of other users on the slaves. A DistributedCache file becomes
+ private by virtue of its permissions on the file system where the
+ files are uploaded, typically HDFS. If the file has no
+ world readable access, or if the directory path leading to the
+ file has no world executable access for lookup, then the file
+ becomes private.
+ </li>
+ <li>"Public" DistributedCache files are cached in a global
+ directory and the file access is setup such that they are
+ publicly visible to all users. These files can be shared by
+ tasks and jobs of all users on the slaves.
+ A DistributedCache file becomes public by virtue of its permissions
+ on the file system where the files are uploaded, typically HDFS.
+ If the file has world readable access, AND if the directory
+ path leading to the file has world executable access for lookup,
+ then the file becomes public. In other words, if the user intends
+ to make a file publicly available to all users, the file permissions
+ must be set to be world readable, and the directory permissions
+ on the path leading to the file must be world executable.
+ </li>
+ </ul>
+ </section>
<p>The <code>DistributedCache</code> tracks modification timestamps
of the cache files/archives. Clearly the cache files/archives should
not be modified by the application or externally