You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Brahma Reddy Battula (JIRA)" <ji...@apache.org> on 2012/05/10 16:58:49 UTC
[jira] [Created] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Brahma Reddy Battula created HDFS-3398:
------------------------------------------
Summary: Client will not retry when primaryDN is down once it's just got pipeline
Key: HDFS-3398
URL: https://issues.apache.org/jira/browse/HDFS-3398
Project: Hadoop HDFS
Issue Type: Task
Affects Versions: 2.0.0
Reporter: Brahma Reddy Battula
Priority: Minor
Fix For: 3.0.0
Scenario:
=========
Start NN and three DN"S
Get the datanode to which blocks has to be replicated.
from
{code}
nodes = nextBlockOutputStream(src);
{code}
Before start writing to the DN ,kill the primary DN.
{code}
// write out data to remote datanode
blockStream.write(buf.array(), buf.position(), buf.remaining());
blockStream.flush();
{code}
Now write will fail with the exception
{noformat}
2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
java.io.IOException: An established connection was aborted by the software in your host machine
at sun.nio.ch.SocketDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(Unknown Source)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
at sun.nio.ch.IOUtil.write(Unknown Source)
at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
at java.io.BufferedOutputStream.write(Unknown Source)
at java.io.DataOutputStream.write(Unknown Source)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
{noformat}
.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284297#comment-13284297 ]
Uma Maheswara Rao G commented on HDFS-3398:
-------------------------------------------
Thanks a lot Amith, for the test steps.
+1 on the latest patch.
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0-alpha
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Attachments: HDFS-3398.patch, HDFS-3398.patch, HDFS_3398_3.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285674#comment-13285674 ]
Hudson commented on HDFS-3398:
------------------------------
Integrated in Hadoop-Mapreduce-trunk #1095 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1095/])
HDFS-3398. Client will not retry when primaryDN is down once it's just got pipeline. Contributed by Amith D K. (Revision 1343944)
Result = SUCCESS
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343944
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0-alpha
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Fix For: 2.0.1-alpha, 3.0.0
>
> Attachments: HDFS-3398.patch, HDFS-3398.patch, HDFS_3398_3.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "Brahma Reddy Battula (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brahma Reddy Battula updated HDFS-3398:
---------------------------------------
Issue Type: Bug (was: Task)
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.0.0
> Reporter: Brahma Reddy Battula
> Priority: Minor
> Fix For: 3.0.0
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "amith (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
amith updated HDFS-3398:
------------------------
Attachment: HDFS-3398.patch
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Attachments: HDFS-3398.patch, HDFS-3398.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HDFS-3398:
------------------------------
Component/s: hdfs client
Target Version/s: 2.0.0
Fix Version/s: (was: 3.0.0)
Does this affect branch-1 as well?
P.S please set "Target version" instead of "Fix version" for unfixed bugs.
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0
> Reporter: Brahma Reddy Battula
> Priority: Minor
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279497#comment-13279497 ]
Hadoop QA commented on HDFS-3398:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12528208/HDFS-3398.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 javadoc. The javadoc tool appears to have generated 2 warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2487//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2487//console
This message is automatically generated.
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Attachments: HDFS-3398.patch, HDFS-3398.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284025#comment-13284025 ]
Hadoop QA commented on HDFS-3398:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12529871/HDFS_3398_3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2525//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2525//console
This message is automatically generated.
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0-alpha
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Attachments: HDFS-3398.patch, HDFS-3398.patch, HDFS_3398_3.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285059#comment-13285059 ]
Hudson commented on HDFS-3398:
------------------------------
Integrated in Hadoop-Common-trunk-Commit #2298 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2298/])
HDFS-3398. Client will not retry when primaryDN is down once it's just got pipeline. Contributed by Amith D K. (Revision 1343944)
Result = SUCCESS
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343944
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0-alpha
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Fix For: 2.0.1-alpha, 3.0.0
>
> Attachments: HDFS-3398.patch, HDFS-3398.patch, HDFS_3398_3.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284041#comment-13284041 ]
Uma Maheswara Rao G commented on HDFS-3398:
-------------------------------------------
Change look good to me. I agree, adding test for this change would be little difficult. Have you tested this manually with debug points?
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0-alpha
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Attachments: HDFS-3398.patch, HDFS-3398.patch, HDFS_3398_3.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uma Maheswara Rao G updated HDFS-3398:
--------------------------------------
Target Version/s: 2.0.0, 3.0.0 (was: 2.0.0)
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0
> Reporter: Brahma Reddy Battula
> Priority: Minor
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "amith (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
amith updated HDFS-3398:
------------------------
Attachment: HDFS-3398.patch
I am submitting the patch removing the
{code}
if (errorIndex == -1)
{code}
check as we have ResponseProcessor to correctly identify the failure DN in the pipeline and to take sutable action.
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0
> Reporter: Brahma Reddy Battula
> Priority: Minor
> Attachments: HDFS-3398.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "amith (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284287#comment-13284287 ]
amith commented on HDFS-3398:
-----------------------------
I manually tested the patch by breakpoints in debug mode
Steps :
1. Put a breakpoint in nodes=nextBlockOutputStream(), blockStream.write(...)
2. Identify the DN primary DN selected, from nodes=nextBlockOutputStream() and when control reach before blockStream.write(...) kill the primary DN
3. Now blockstream which is pointing to primary DN will not be able to send data so IOException will be thrown
Result Without patch :
Since in the catch block haserror is set and no errorIndex so we treat is as a client error and not DN error so client will stop.
Result with patch :
we are handling the IOException from the blockstream and set errorindex to primary DN and rethrowing the exception we have both errorIndex=0 and hasError=true so this is treated as DN failure not clirnt failure so client will try to update its pipeline, and continue writing.
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0-alpha
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Attachments: HDFS-3398.patch, HDFS-3398.patch, HDFS_3398_3.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272598#comment-13272598 ]
Uma Maheswara Rao G commented on HDFS-3398:
-------------------------------------------
Seems to be a good catch Brahma.
@Todd, It looks to be problem to me Todd. When writing on to socket if other peer goes down, it may treat that as client error and client will exit.
How about catching socket operations and setting errorIndex to 1 (treating first node as bad)?
I did not see the below check in 205 code.
{code}
if (errorIndex == -1) { // not a datanode error
streamerClosed = true;
}
{code}
205 code on throwable:
{code}
} catch (Throwable e) {
LOG.warn("DataStreamer Exception: " +
StringUtils.stringifyException(e));
if (e instanceof IOException) {
setLastException((IOException)e);
}
hasError = true;
}
}
{code}
In trunk:
{code}
} catch (Throwable e) {
DFSClient.LOG.warn("DataStreamer Exception", e);
if (e instanceof IOException) {
setLastException((IOException)e);
}
hasError = true;
if (errorIndex == -1) { // not a datanode error
streamerClosed = true;
}
}
{code}
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0
> Reporter: Brahma Reddy Battula
> Priority: Minor
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285055#comment-13285055 ]
Uma Maheswara Rao G commented on HDFS-3398:
-------------------------------------------
I have just committed this to trunk and barnch-2. Thanks a lot, Amith for the patch.
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0-alpha
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Fix For: 2.0.1-alpha, 3.0.0
>
> Attachments: HDFS-3398.patch, HDFS-3398.patch, HDFS_3398_3.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285057#comment-13285057 ]
Hudson commented on HDFS-3398:
------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2371 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2371/])
HDFS-3398. Client will not retry when primaryDN is down once it's just got pipeline. Contributed by Amith D K. (Revision 1343944)
Result = SUCCESS
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343944
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0-alpha
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Fix For: 2.0.1-alpha, 3.0.0
>
> Attachments: HDFS-3398.patch, HDFS-3398.patch, HDFS_3398_3.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285098#comment-13285098 ]
Hudson commented on HDFS-3398:
------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2317 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2317/])
HDFS-3398. Client will not retry when primaryDN is down once it's just got pipeline. Contributed by Amith D K. (Revision 1343944)
Result = FAILURE
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343944
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0-alpha
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Fix For: 2.0.1-alpha, 3.0.0
>
> Attachments: HDFS-3398.patch, HDFS-3398.patch, HDFS_3398_3.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HDFS-3398) Client will not retry when primaryDN is
down once it's just got pipeline
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy closed HDFS-3398.
-------------------------------
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0-alpha
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Fix For: 3.0.0, 2.0.2-alpha
>
> Attachments: HDFS_3398_3.patch, HDFS-3398.patch, HDFS-3398.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "amith (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
amith reassigned HDFS-3398:
---------------------------
Assignee: amith
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Attachments: HDFS-3398.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "amith (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
amith updated HDFS-3398:
------------------------
Status: Patch Available (was: Open)
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Attachments: HDFS-3398.patch, HDFS-3398.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uma Maheswara Rao G updated HDFS-3398:
--------------------------------------
Resolution: Fixed
Fix Version/s: 3.0.0
2.0.1-alpha
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0-alpha
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Fix For: 2.0.1-alpha, 3.0.0
>
> Attachments: HDFS-3398.patch, HDFS-3398.patch, HDFS_3398_3.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "amith (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
amith updated HDFS-3398:
------------------------
Attachment: HDFS_3398_3.patch
Attaching the patch
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0-alpha
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Attachments: HDFS-3398.patch, HDFS-3398.patch, HDFS_3398_3.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "amith (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284029#comment-13284029 ]
amith commented on HDFS-3398:
-----------------------------
I am not able to write test since it require the primary DN down before blockstream.write(...) is called.
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0-alpha
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Attachments: HDFS-3398.patch, HDFS-3398.patch, HDFS_3398_3.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3398) Client will not retry when primaryDN
is down once it's just got pipeline
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285592#comment-13285592 ]
Hudson commented on HDFS-3398:
------------------------------
Integrated in Hadoop-Hdfs-trunk #1061 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1061/])
HDFS-3398. Client will not retry when primaryDN is down once it's just got pipeline. Contributed by Amith D K. (Revision 1343944)
Result = SUCCESS
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343944
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
> Client will not retry when primaryDN is down once it's just got pipeline
> ------------------------------------------------------------------------
>
> Key: HDFS-3398
> URL: https://issues.apache.org/jira/browse/HDFS-3398
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0-alpha
> Reporter: Brahma Reddy Battula
> Assignee: amith
> Priority: Minor
> Fix For: 2.0.1-alpha, 3.0.0
>
> Attachments: HDFS-3398.patch, HDFS-3398.patch, HDFS_3398_3.patch
>
>
> Scenario:
> =========
> Start NN and three DN"S
> Get the datanode to which blocks has to be replicated.
> from
> {code}
> nodes = nextBlockOutputStream(src);
> {code}
> Before start writing to the DN ,kill the primary DN.
> {code}
> // write out data to remote datanode
> blockStream.write(buf.array(), buf.position(), buf.remaining());
> blockStream.flush();
> {code}
> Now write will fail with the exception
> {noformat}
> 2012-05-10 14:21:47,993 WARN hdfs.DFSClient (DFSOutputStream.java:run(552)) - DataStreamer Exception
> java.io.IOException: An established connection was aborted by the software in your host machine
> at sun.nio.ch.SocketDispatcher.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:60)
> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:151)
> at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:112)
> at java.io.BufferedOutputStream.write(Unknown Source)
> at java.io.DataOutputStream.write(Unknown Source)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:513)
> {noformat}
> .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira