You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by dd...@apache.org on 2008/10/07 08:55:30 UTC
svn commit: r702360 [1/3] - in /hadoop/core/trunk: ./ docs/
src/docs/src/documentation/content/xdocs/
src/mapred/org/apache/hadoop/mapred/ src/test/org/apache/hadoop/mapred/
src/webapps/job/
Author: ddas
Date: Mon Oct 6 23:55:29 2008
New Revision: 702360
URL: http://svn.apache.org/viewvc?rev=702360&view=rev
Log:
HADOOP-4261. Adds a setup task for jobs. This is required so that we don't setup jobs that haven't been inited yet (since init could lead to job failure). Only after the init has successfully happened do we launch the setupJob task. Contributed by Amareshwari Sriramadasu.
Modified:
hadoop/core/trunk/CHANGES.txt
hadoop/core/trunk/docs/changes.html
hadoop/core/trunk/docs/mapred_tutorial.html
hadoop/core/trunk/docs/mapred_tutorial.pdf
hadoop/core/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml
hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/EagerTaskInitializationListener.java
hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/HistoryViewer.java
hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobClient.java
hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobHistory.java
hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobInProgress.java
hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobStatus.java
hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobSubmissionProtocol.java
hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobTracker.java
hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/LocalJobRunner.java
hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/MapTask.java
hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/OutputCommitter.java
hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/ReduceTask.java
hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/RunningJob.java
hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/Task.java
hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/TaskInProgress.java
hadoop/core/trunk/src/test/org/apache/hadoop/mapred/MiniMRCluster.java
hadoop/core/trunk/src/test/org/apache/hadoop/mapred/TestJobTrackerRestart.java
hadoop/core/trunk/src/test/org/apache/hadoop/mapred/TestMiniMRLocalFS.java
hadoop/core/trunk/src/webapps/job/jobdetails.jsp
hadoop/core/trunk/src/webapps/job/jobdetailshistory.jsp
hadoop/core/trunk/src/webapps/job/jobtasks.jsp
hadoop/core/trunk/src/webapps/job/taskdetails.jsp
Modified: hadoop/core/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/core/trunk/CHANGES.txt?rev=702360&r1=702359&r2=702360&view=diff
==============================================================================
--- hadoop/core/trunk/CHANGES.txt (original)
+++ hadoop/core/trunk/CHANGES.txt Mon Oct 6 23:55:29 2008
@@ -859,6 +859,11 @@
HADOOP-4163. Report FSErrors from map output fetch threads instead of
merely logging them. (Sharad Agarwal via cdouglas)
+ HADOOP-4261. Adds a setup task for jobs. This is required so that we
+ don't setup jobs that haven't been inited yet (since init could lead
+ to job failure). Only after the init has successfully happened do we
+ launch the setupJob task. (Amareshwari Sriramadasu via ddas)
+
Release 0.18.2 - Unreleased
BUG FIXES
Modified: hadoop/core/trunk/docs/changes.html
URL: http://svn.apache.org/viewvc/hadoop/core/trunk/docs/changes.html?rev=702360&r1=702359&r2=702360&view=diff
==============================================================================
--- hadoop/core/trunk/docs/changes.html (original)
+++ hadoop/core/trunk/docs/changes.html Mon Oct 6 23:55:29 2008
@@ -259,7 +259,7 @@
</ol>
</li>
<li><a href="javascript:toggleList('release_0.19.0_-_unreleased_._improvements_')"> IMPROVEMENTS
-</a> (70)
+</a> (72)
<ol id="release_0.19.0_-_unreleased_._improvements_">
<li><a href="http://issues.apache.org/jira/browse/HADOOP-4205">HADOOP-4205</a>. hive: metastore and ql to use the refactored SerDe library.<br />(zshao)</li>
<li><a href="http://issues.apache.org/jira/browse/HADOOP-4106">HADOOP-4106</a>. libhdfs: add time, permission and user attribute support
@@ -387,6 +387,8 @@
make itermitant failures reproducible.<br />(szetszwo via cdouglas)</li>
<li><a href="http://issues.apache.org/jira/browse/HADOOP-4209">HADOOP-4209</a>. Remove the change to the format of task attempt id by
incrementing the task attempt numbers by 1000 when the job restarts.<br />(Amar Kamat via omalley)</li>
+ <li><a href="http://issues.apache.org/jira/browse/HADOOP-4301">HADOOP-4301</a>. Adds forrest doc for the skip bad records feature.<br />(Sharad Agarwal via ddas)</li>
+ <li><a href="http://issues.apache.org/jira/browse/HADOOP-4354">HADOOP-4354</a>. Separate TestDatanodeDeath.testDatanodeDeath() into 4 tests.<br />(szetszwo)</li>
</ol>
</li>
<li><a href="javascript:toggleList('release_0.19.0_-_unreleased_._optimizations_')"> OPTIMIZATIONS
@@ -413,7 +415,7 @@
</ol>
</li>
<li><a href="javascript:toggleList('release_0.19.0_-_unreleased_._bug_fixes_')"> BUG FIXES
-</a> (108)
+</a> (109)
<ol id="release_0.19.0_-_unreleased_._bug_fixes_">
<li><a href="http://issues.apache.org/jira/browse/HADOOP-3563">HADOOP-3563</a>. Refactor the distributed upgrade code so that it is
easier to identify datanode and namenode related code.<br />(dhruba)</li>
@@ -621,6 +623,8 @@
retries for fetching map-outputs; also fixed the case where the reducer
automatically kills on too many unique map-outputs could not be fetched
for small jobs.<br />(Amareshwari Sri Ramadasu via acmurthy)</li>
+ <li><a href="http://issues.apache.org/jira/browse/HADOOP-4163">HADOOP-4163</a>. Report FSErrors from map output fetch threads instead of
+merely logging them.<br />(Sharad Agarwal via cdouglas)</li>
</ol>
</li>
</ul>
Modified: hadoop/core/trunk/docs/mapred_tutorial.html
URL: http://svn.apache.org/viewvc/hadoop/core/trunk/docs/mapred_tutorial.html?rev=702360&r1=702359&r2=702360&view=diff
==============================================================================
--- hadoop/core/trunk/docs/mapred_tutorial.html (original)
+++ hadoop/core/trunk/docs/mapred_tutorial.html Mon Oct 6 23:55:29 2008
@@ -2186,14 +2186,12 @@
<li>
Setup the job during initialization. For example, create
the temporary output directory for the job during the
- initialization of the job. The job client does the setup
- for the job.
+ initialization of the job.
</li>
<li>
Cleanup the job after the job completion. For example, remove the
- temporary output directory after the job completion. A separate
- task does the cleanupJob at the end of the job.
+ temporary output directory after the job completion.
</li>
<li>