You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by cu...@apache.org on 2006/06/21 20:35:32 UTC
svn commit: r416062 - in /lucene/hadoop/trunk: CHANGES.txt
src/java/org/apache/hadoop/dfs/DFSClient.java
Author: cutting
Date: Wed Jun 21 11:35:32 2006
New Revision: 416062
URL: http://svn.apache.org/viewvc?rev=416062&view=rev
Log:
HADOOP-311. Change DFS client to retry failed reads, so that a single read failure alone will not cause failure of a task. Contributed by Owen.
Modified:
lucene/hadoop/trunk/CHANGES.txt
lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java
Modified: lucene/hadoop/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/lucene/hadoop/trunk/CHANGES.txt?rev=416062&r1=416061&r2=416062&view=diff
==============================================================================
--- lucene/hadoop/trunk/CHANGES.txt (original)
+++ lucene/hadoop/trunk/CHANGES.txt Wed Jun 21 11:35:32 2006
@@ -26,6 +26,10 @@
protocol: clients and servers must both be upgraded to the new
version to ensure correct operation. (Devaraj Das via cutting)
+ 7. HADOOP-311. Change DFS client to retry failed reads, so that a
+ single read failure will not alone cause failure of a task.
+ (omalley via cutting)
+
Release 0.3.2 - 2006-06-09
Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java
URL: http://svn.apache.org/viewvc/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java?rev=416062&r1=416061&r2=416062&view=diff
==============================================================================
--- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java (original)
+++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java Wed Jun 21 11:35:32 2006
@@ -634,14 +634,26 @@
throw new IOException("Stream closed");
}
if (pos < filelen) {
- if (pos > blockEnd) {
- blockSeekTo(pos);
+ int retries = 2;
+ while (retries > 0) {
+ try {
+ if (pos > blockEnd) {
+ blockSeekTo(pos);
+ }
+ int realLen = Math.min(len, (int) (blockEnd - pos + 1));
+ int result = blockStream.read(buf, off, realLen);
+ if (result >= 0) {
+ pos += result;
+ }
+ return result;
+ } catch (IOException e) {
+ LOG.warn("DFS Read: " + StringUtils.stringifyException(e));
+ blockEnd = -1;
+ if (--retries == 0) {
+ throw e;
+ }
}
- int result = blockStream.read(buf, off, Math.min(len, (int) (blockEnd - pos + 1)));
- if (result >= 0) {
- pos += result;
- }
- return result;
+ }
}
return -1;
}