You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by dd...@apache.org on 2008/06/17 07:42:23 UTC
svn commit: r668401 [3/3] - in /hadoop/core/branches/branch-0.18:
CHANGES.txt docs/mapred_tutorial.html docs/mapred_tutorial.pdf
src/docs/src/documentation/content/xdocs/mapred_tutorial.xml
src/docs/src/documentation/content/xdocs/site.xml
Modified: hadoop/core/branches/branch-0.18/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml
URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.18/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml?rev=668401&r1=668400&r2=668401&view=diff
==============================================================================
--- hadoop/core/branches/branch-0.18/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml (original)
+++ hadoop/core/branches/branch-0.18/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml Mon Jun 16 22:42:23 2008
@@ -1562,6 +1562,43 @@
<p><code>IsolationRunner</code> will run the failed task in a single
jvm, which can be in the debugger, over precisely the same input.</p>
</section>
+
+ <section>
+ <title>Profiling</title>
+ <p>Profiling is a utility to get a representative (2 or 3) sample
+ of built-in java profiler for a sample of maps and reduces. </p>
+
+ <p>User can specify whether the system should collect profiler
+ information for some of the tasks in the job by setting the
+ configuration property <code>mapred.task.profile</code>. The
+ value can be set using the api
+ <a href="ext:api/org/apache/hadoop/mapred/jobconf/setprofileenabled">
+ JobConf.setProfileEnabled(boolean)</a>. If the value is set
+ <code>true</code>, the task profiling is enabled. The profiler
+ information is stored in the the user log directory. By default,
+ profiling is not enabled for the job. </p>
+
+ <p>Once user configures that profiling is needed, she/he can use
+ the configuration property
+ <code>mapred.task.profile.{maps|reduces}</code> to set the ranges
+ of map/reduce tasks to profile. The value can be set using the api
+ <a href="ext:api/org/apache/hadoop/mapred/jobconf/setprofiletaskrange">
+ JobConf.setProfileTaskRange(boolean,String)</a>.
+ By default, the specified range is <code>0-2</code>.</p>
+
+ <p>User can also specify the profiler configuration arguments by
+ setting the configuration property
+ <code>mapred.task.profile.params</code>. The value can be specified
+ using the api
+ <a href="ext:api/org/apache/hadoop/mapred/jobconf/setprofileparams">
+ JobConf.setProfileParams(String)</a>. If the string contains a
+ <code>%s</code>, it will be replaced with the name of the profiling
+ output file when the task runs. These parameters are passed to the
+ task child JVM on the command line. The default value for
+ the profiling parameters is
+ <code>-agentlib:hprof=cpu=samples,heap=sites,force=n,thread=y,verbose=n,file=%s</code>
+ </p>
+ </section>
<section>
<title>Debugging</title>
Modified: hadoop/core/branches/branch-0.18/src/docs/src/documentation/content/xdocs/site.xml
URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.18/src/docs/src/documentation/content/xdocs/site.xml?rev=668401&r1=668400&r2=668401&view=diff
==============================================================================
--- hadoop/core/branches/branch-0.18/src/docs/src/documentation/content/xdocs/site.xml (original)
+++ hadoop/core/branches/branch-0.18/src/docs/src/documentation/content/xdocs/site.xml Mon Jun 16 22:42:23 2008
@@ -166,6 +166,9 @@
<setjobendnotificationuri href="#setJobEndNotificationURI(java.lang.String)" />
<setcompressmapoutput href="#setCompressMapOutput(boolean)" />
<setmapoutputcompressorclass href="#setMapOutputCompressorClass(java.lang.Class)" />
+ <setprofileenabled href="#setProfileEnabled(boolean)" />
+ <setprofiletaskrange href="#setProfileTaskRange(boolean,%20java.lang.String)" />
+ <setprofileparams href="#setProfileParams(java.lang.String)" />
<getjoblocaldir href="#getJobLocalDir()" />
<getjar href="#getJar()" />
</jobconf>