svn commit: r606855 - in /lucene/hadoop/branches/branch-0.15: CHANGES.txt src/java/org/apache/hadoop/mapred/ReduceTask.java

[dd...@apache.org]

Author: ddas
Date: Wed Dec 26 00:39:56 2007
New Revision: 606855

URL: http://svn.apache.org/viewvc?rev=606855&view=rev
Log:
Merge -r 606853:606854 from trunk to 0.15 branch. Fixes HADOOP-2486.

Modified:
    lucene/hadoop/branches/branch-0.15/CHANGES.txt
    lucene/hadoop/branches/branch-0.15/src/java/org/apache/hadoop/mapred/ReduceTask.java

Modified: lucene/hadoop/branches/branch-0.15/CHANGES.txt
URL: http://svn.apache.org/viewvc/lucene/hadoop/branches/branch-0.15/CHANGES.txt?rev=606855&r1=606854&r2=606855&view=diff
==============================================================================
--- lucene/hadoop/branches/branch-0.15/CHANGES.txt (original)
+++ lucene/hadoop/branches/branch-0.15/CHANGES.txt Wed Dec 26 00:39:56 2007
@@ -39,6 +39,10 @@
     round-robin disk selections randomly. This helps in spreading data across
     multiple partitions much better. (acmurhty)
 
+    HADOOP-2486. When the list of files from the InMemoryFileSystem is obtained
+    for merging, this patch will ensure that only those files whose checksums
+    have also got created (renamed) are returned. (ddas)
+
   IMPROVEMENTS
 
     HADOOP-2160.  Remove project-level, non-user documentation from

Modified: lucene/hadoop/branches/branch-0.15/src/java/org/apache/hadoop/mapred/ReduceTask.java
URL: http://svn.apache.org/viewvc/lucene/hadoop/branches/branch-0.15/src/java/org/apache/hadoop/mapred/ReduceTask.java?rev=606855&r1=606854&r2=606855&view=diff
==============================================================================
--- lucene/hadoop/branches/branch-0.15/src/java/org/apache/hadoop/mapred/ReduceTask.java (original)
+++ lucene/hadoop/branches/branch-0.15/src/java/org/apache/hadoop/mapred/ReduceTask.java Wed Dec 26 00:39:56 2007
@@ -1290,8 +1290,11 @@
       public void run() {
         LOG.info(reduceTask.getTaskId() + " Thread started: " + getName());
         try {
+          Path[] inMemClosedFiles;
           //initiate merge
-          Path[] inMemClosedFiles = inMemFileSys.getFiles(MAP_OUTPUT_FILTER);
+          synchronized (ReduceTask.this) {
+            inMemClosedFiles = inMemFileSys.getFiles(MAP_OUTPUT_FILTER);
+          }
           //Note that the above Path[] could be of length 0 if all copies are 
           //in flight. So we make sure that we have some 'closed' map
           //output files to merge to get the benefit of in-memory merge