You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by cu...@apache.org on 2007/05/14 20:09:30 UTC
svn commit: r537928 - in /lucene/hadoop/trunk: CHANGES.txt
src/java/org/apache/hadoop/mapred/TaskTracker.java
Author: cutting
Date: Mon May 14 11:09:30 2007
New Revision: 537928
URL: http://svn.apache.org/viewvc?view=rev&rev=537928
Log:
HADOOP-1350. Fix shuffle performance problem caused by forcing chunked encoding of map outputs. Contributed by Devaraj.
Modified:
lucene/hadoop/trunk/CHANGES.txt
lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/TaskTracker.java
Modified: lucene/hadoop/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/lucene/hadoop/trunk/CHANGES.txt?view=diff&rev=537928&r1=537927&r2=537928
==============================================================================
--- lucene/hadoop/trunk/CHANGES.txt (original)
+++ lucene/hadoop/trunk/CHANGES.txt Mon May 14 11:09:30 2007
@@ -386,6 +386,9 @@
110. HADOOP-1341. Fix intermittent failures in HBase unit tests
caused by deadlock. (Jim Kellerman via cutting)
+111. HADOOP-1350. Fix shuffle performance problem caused by forcing
+ chunked encoding of map outputs. (Devaraj Das via cutting)
+
Release 0.12.3 - 2007-04-06
Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/TaskTracker.java
URL: http://svn.apache.org/viewvc/lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/TaskTracker.java?view=diff&rev=537928&r1=537927&r2=537928
==============================================================================
--- lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/TaskTracker.java (original)
+++ lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/TaskTracker.java Mon May 14 11:09:30 2007
@@ -1920,9 +1920,6 @@
//the actual number of bytes being transferred
response.setHeader(MAP_OUTPUT_LENGTH, Long.toString(partLength));
- //use 'chunked' transfer-encoding for transferring data
- response.setHeader("Transfer-Encoding", "chunked");
-
//use the same buffersize as used for reading the data from disk
response.setBufferSize(MAX_BYTES_TO_READ);
@@ -1942,6 +1939,7 @@
while (len > 0) {
try {
outStream.write(buffer, 0, len);
+ outStream.flush();
} catch (IOException ie) {
isInputException = false;
throw ie;