You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Konstantin Shvachko (JIRA)" <ji...@apache.org> on 2007/10/09 23:25:50 UTC
[jira] Created: (HADOOP-2018) Broken pipe SocketException in
DataNode$DataXceiver
Broken pipe SocketException in DataNode$DataXceiver
---------------------------------------------------
Key: HADOOP-2018
URL: https://issues.apache.org/jira/browse/HADOOP-2018
Project: Hadoop
Issue Type: Bug
Components: dfs
Affects Versions: 0.15.0
Reporter: Konstantin Shvachko
Priority: Blocker
Fix For: 0.15.0
I have 2 data-nodes, one of which is trying to replicate blocks to another.
The second data-node throws the following excpetion for every replicated block.
{code}
07/10/09 20:36:39 INFO dfs.DataNode: Received block blk_-8942388986043611634 from /a.d.d.r:43159
07/10/09 20:36:39 WARN dfs.DataNode: Error writing reply back to /a.d.d.r:43159for writing block blk_-8942388986043611634
07/10/09 20:36:39 WARN dfs.DataNode: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
at java.io.DataOutputStream.writeShort(DataOutputStream.java:151)
at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:939)
at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
at java.lang.Thread.run(Thread.java:619)
{code}
# It looks like that the first data-node does not expect to receive anything from the second one and closes the connection.
# There should be a space in front of
{code}
+ "for writing block " + block );
{code}
# The port number is misleading in these messages. DataXceivers open sockets on different ports every time, which is
different from the data-node's main port. So we should rather print here the main port in order to be able to recognize
wich data-node the block was sent from.
Is this related to HADOOP-1908?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2018) Broken pipe SocketException in
DataNode$DataXceiver
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534110 ]
Hadoop QA commented on HADOOP-2018:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12367583/pipe1.patch
against trunk revision r583839.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests -1. The patch failed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/926/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/926/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/926/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/926/console
This message is automatically generated.
> Broken pipe SocketException in DataNode$DataXceiver
> ---------------------------------------------------
>
> Key: HADOOP-2018
> URL: https://issues.apache.org/jira/browse/HADOOP-2018
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.0
> Reporter: Konstantin Shvachko
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.15.0
>
> Attachments: pipe.patch, pipe1.patch
>
>
> I have 2 data-nodes, one of which is trying to replicate blocks to another.
> The second data-node throws the following excpetion for every replicated block.
> {code}
> 07/10/09 20:36:39 INFO dfs.DataNode: Received block blk_-8942388986043611634 from /a.d.d.r:43159
> 07/10/09 20:36:39 WARN dfs.DataNode: Error writing reply back to /a.d.d.r:43159for writing block blk_-8942388986043611634
> 07/10/09 20:36:39 WARN dfs.DataNode: java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
> at java.io.DataOutputStream.writeShort(DataOutputStream.java:151)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:939)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> # It looks like that the first data-node does not expect to receive anything from the second one and closes the connection.
> # There should be a space in front of
> {code}
> + "for writing block " + block );
> {code}
> # The port number is misleading in these messages. DataXceivers open sockets on different ports every time, which is
> different from the data-node's main port. So we should rather print here the main port in order to be able to recognize
> wich data-node the block was sent from.
> Is this related to HADOOP-1908?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-2018) Broken pipe SocketException in
DataNode$DataXceiver
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang reassigned HADOOP-2018:
-------------------------------------
Assignee: Hairong Kuang
> Broken pipe SocketException in DataNode$DataXceiver
> ---------------------------------------------------
>
> Key: HADOOP-2018
> URL: https://issues.apache.org/jira/browse/HADOOP-2018
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.0
> Reporter: Konstantin Shvachko
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.15.0
>
>
> I have 2 data-nodes, one of which is trying to replicate blocks to another.
> The second data-node throws the following excpetion for every replicated block.
> {code}
> 07/10/09 20:36:39 INFO dfs.DataNode: Received block blk_-8942388986043611634 from /a.d.d.r:43159
> 07/10/09 20:36:39 WARN dfs.DataNode: Error writing reply back to /a.d.d.r:43159for writing block blk_-8942388986043611634
> 07/10/09 20:36:39 WARN dfs.DataNode: java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
> at java.io.DataOutputStream.writeShort(DataOutputStream.java:151)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:939)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> # It looks like that the first data-node does not expect to receive anything from the second one and closes the connection.
> # There should be a space in front of
> {code}
> + "for writing block " + block );
> {code}
> # The port number is misleading in these messages. DataXceivers open sockets on different ports every time, which is
> different from the data-node's main port. So we should rather print here the main port in order to be able to recognize
> wich data-node the block was sent from.
> Is this related to HADOOP-1908?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-2018) Broken pipe
SocketException in DataNode$DataXceiver
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533533 ]
hairong edited comment on HADOOP-2018 at 10/10/07 10:46 AM:
------------------------------------------------------------------
yes, the bug is fixed in HADOOP-1912. But since HADOOP-1912 is not in release 15, I will submit a seprate patch for this.
was (Author: hairong):
yes, the bug is fixed in /HADOOP-1912. But since /HADOOP-1912 is not in release 15, I will submit a seprate patch for this.
> Broken pipe SocketException in DataNode$DataXceiver
> ---------------------------------------------------
>
> Key: HADOOP-2018
> URL: https://issues.apache.org/jira/browse/HADOOP-2018
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.0
> Reporter: Konstantin Shvachko
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.15.0
>
>
> I have 2 data-nodes, one of which is trying to replicate blocks to another.
> The second data-node throws the following excpetion for every replicated block.
> {code}
> 07/10/09 20:36:39 INFO dfs.DataNode: Received block blk_-8942388986043611634 from /a.d.d.r:43159
> 07/10/09 20:36:39 WARN dfs.DataNode: Error writing reply back to /a.d.d.r:43159for writing block blk_-8942388986043611634
> 07/10/09 20:36:39 WARN dfs.DataNode: java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
> at java.io.DataOutputStream.writeShort(DataOutputStream.java:151)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:939)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> # It looks like that the first data-node does not expect to receive anything from the second one and closes the connection.
> # There should be a space in front of
> {code}
> + "for writing block " + block );
> {code}
> # The port number is misleading in these messages. DataXceivers open sockets on different ports every time, which is
> different from the data-node's main port. So we should rather print here the main port in order to be able to recognize
> wich data-node the block was sent from.
> Is this related to HADOOP-1908?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2018) Broken pipe SocketException in
DataNode$DataXceiver
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-2018:
----------------------------------
Attachment: pipe.patch
> Broken pipe SocketException in DataNode$DataXceiver
> ---------------------------------------------------
>
> Key: HADOOP-2018
> URL: https://issues.apache.org/jira/browse/HADOOP-2018
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.0
> Reporter: Konstantin Shvachko
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.15.0
>
> Attachments: pipe.patch
>
>
> I have 2 data-nodes, one of which is trying to replicate blocks to another.
> The second data-node throws the following excpetion for every replicated block.
> {code}
> 07/10/09 20:36:39 INFO dfs.DataNode: Received block blk_-8942388986043611634 from /a.d.d.r:43159
> 07/10/09 20:36:39 WARN dfs.DataNode: Error writing reply back to /a.d.d.r:43159for writing block blk_-8942388986043611634
> 07/10/09 20:36:39 WARN dfs.DataNode: java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
> at java.io.DataOutputStream.writeShort(DataOutputStream.java:151)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:939)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> # It looks like that the first data-node does not expect to receive anything from the second one and closes the connection.
> # There should be a space in front of
> {code}
> + "for writing block " + block );
> {code}
> # The port number is misleading in these messages. DataXceivers open sockets on different ports every time, which is
> different from the data-node's main port. So we should rather print here the main port in order to be able to recognize
> wich data-node the block was sent from.
> Is this related to HADOOP-1908?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2018) Broken pipe SocketException in
DataNode$DataXceiver
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534156 ]
dhruba borthakur commented on HADOOP-2018:
------------------------------------------
Although the source datanode now waits for a response from the target datanode before closing the stream, it should not affect replication-data-transfer-throughout for the cluster.
> Broken pipe SocketException in DataNode$DataXceiver
> ---------------------------------------------------
>
> Key: HADOOP-2018
> URL: https://issues.apache.org/jira/browse/HADOOP-2018
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.0
> Reporter: Konstantin Shvachko
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.15.0
>
> Attachments: pipe.patch, pipe1.patch
>
>
> I have 2 data-nodes, one of which is trying to replicate blocks to another.
> The second data-node throws the following excpetion for every replicated block.
> {code}
> 07/10/09 20:36:39 INFO dfs.DataNode: Received block blk_-8942388986043611634 from /a.d.d.r:43159
> 07/10/09 20:36:39 WARN dfs.DataNode: Error writing reply back to /a.d.d.r:43159for writing block blk_-8942388986043611634
> 07/10/09 20:36:39 WARN dfs.DataNode: java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
> at java.io.DataOutputStream.writeShort(DataOutputStream.java:151)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:939)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> # It looks like that the first data-node does not expect to receive anything from the second one and closes the connection.
> # There should be a space in front of
> {code}
> + "for writing block " + block );
> {code}
> # The port number is misleading in these messages. DataXceivers open sockets on different ports every time, which is
> different from the data-node's main port. So we should rather print here the main port in order to be able to recognize
> wich data-node the block was sent from.
> Is this related to HADOOP-1908?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2018) Broken pipe SocketException in
DataNode$DataXceiver
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-2018:
----------------------------------
Status: Patch Available (was: Open)
> Broken pipe SocketException in DataNode$DataXceiver
> ---------------------------------------------------
>
> Key: HADOOP-2018
> URL: https://issues.apache.org/jira/browse/HADOOP-2018
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.0
> Reporter: Konstantin Shvachko
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.15.0
>
> Attachments: pipe.patch, pipe1.patch
>
>
> I have 2 data-nodes, one of which is trying to replicate blocks to another.
> The second data-node throws the following excpetion for every replicated block.
> {code}
> 07/10/09 20:36:39 INFO dfs.DataNode: Received block blk_-8942388986043611634 from /a.d.d.r:43159
> 07/10/09 20:36:39 WARN dfs.DataNode: Error writing reply back to /a.d.d.r:43159for writing block blk_-8942388986043611634
> 07/10/09 20:36:39 WARN dfs.DataNode: java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
> at java.io.DataOutputStream.writeShort(DataOutputStream.java:151)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:939)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> # It looks like that the first data-node does not expect to receive anything from the second one and closes the connection.
> # There should be a space in front of
> {code}
> + "for writing block " + block );
> {code}
> # The port number is misleading in these messages. DataXceivers open sockets on different ports every time, which is
> different from the data-node's main port. So we should rather print here the main port in order to be able to recognize
> wich data-node the block was sent from.
> Is this related to HADOOP-1908?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2018) Broken pipe SocketException in
DataNode$DataXceiver
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534119 ]
Konstantin Shvachko commented on HADOOP-2018:
---------------------------------------------
+1
hbase test failure is not related to the patch.
> Broken pipe SocketException in DataNode$DataXceiver
> ---------------------------------------------------
>
> Key: HADOOP-2018
> URL: https://issues.apache.org/jira/browse/HADOOP-2018
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.0
> Reporter: Konstantin Shvachko
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.15.0
>
> Attachments: pipe.patch, pipe1.patch
>
>
> I have 2 data-nodes, one of which is trying to replicate blocks to another.
> The second data-node throws the following excpetion for every replicated block.
> {code}
> 07/10/09 20:36:39 INFO dfs.DataNode: Received block blk_-8942388986043611634 from /a.d.d.r:43159
> 07/10/09 20:36:39 WARN dfs.DataNode: Error writing reply back to /a.d.d.r:43159for writing block blk_-8942388986043611634
> 07/10/09 20:36:39 WARN dfs.DataNode: java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
> at java.io.DataOutputStream.writeShort(DataOutputStream.java:151)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:939)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> # It looks like that the first data-node does not expect to receive anything from the second one and closes the connection.
> # There should be a space in front of
> {code}
> + "for writing block " + block );
> {code}
> # The port number is misleading in these messages. DataXceivers open sockets on different ports every time, which is
> different from the data-node's main port. So we should rather print here the main port in order to be able to recognize
> wich data-node the block was sent from.
> Is this related to HADOOP-1908?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2018) Broken pipe SocketException in
DataNode$DataXceiver
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-2018:
----------------------------------
Attachment: pipe1.patch
The patch fixed the leading space problem.
> Broken pipe SocketException in DataNode$DataXceiver
> ---------------------------------------------------
>
> Key: HADOOP-2018
> URL: https://issues.apache.org/jira/browse/HADOOP-2018
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.0
> Reporter: Konstantin Shvachko
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.15.0
>
> Attachments: pipe.patch, pipe1.patch
>
>
> I have 2 data-nodes, one of which is trying to replicate blocks to another.
> The second data-node throws the following excpetion for every replicated block.
> {code}
> 07/10/09 20:36:39 INFO dfs.DataNode: Received block blk_-8942388986043611634 from /a.d.d.r:43159
> 07/10/09 20:36:39 WARN dfs.DataNode: Error writing reply back to /a.d.d.r:43159for writing block blk_-8942388986043611634
> 07/10/09 20:36:39 WARN dfs.DataNode: java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
> at java.io.DataOutputStream.writeShort(DataOutputStream.java:151)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:939)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> # It looks like that the first data-node does not expect to receive anything from the second one and closes the connection.
> # There should be a space in front of
> {code}
> + "for writing block " + block );
> {code}
> # The port number is misleading in these messages. DataXceivers open sockets on different ports every time, which is
> different from the data-node's main port. So we should rather print here the main port in order to be able to recognize
> wich data-node the block was sent from.
> Is this related to HADOOP-1908?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2018) Broken pipe SocketException in
DataNode$DataXceiver
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-2018:
-------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thank Hairong!
> Broken pipe SocketException in DataNode$DataXceiver
> ---------------------------------------------------
>
> Key: HADOOP-2018
> URL: https://issues.apache.org/jira/browse/HADOOP-2018
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.0
> Reporter: Konstantin Shvachko
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.15.0
>
> Attachments: pipe.patch, pipe1.patch
>
>
> I have 2 data-nodes, one of which is trying to replicate blocks to another.
> The second data-node throws the following excpetion for every replicated block.
> {code}
> 07/10/09 20:36:39 INFO dfs.DataNode: Received block blk_-8942388986043611634 from /a.d.d.r:43159
> 07/10/09 20:36:39 WARN dfs.DataNode: Error writing reply back to /a.d.d.r:43159for writing block blk_-8942388986043611634
> 07/10/09 20:36:39 WARN dfs.DataNode: java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
> at java.io.DataOutputStream.writeShort(DataOutputStream.java:151)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:939)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> # It looks like that the first data-node does not expect to receive anything from the second one and closes the connection.
> # There should be a space in front of
> {code}
> + "for writing block " + block );
> {code}
> # The port number is misleading in these messages. DataXceivers open sockets on different ports every time, which is
> different from the data-node's main port. So we should rather print here the main port in order to be able to recognize
> wich data-node the block was sent from.
> Is this related to HADOOP-1908?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2018) Broken pipe SocketException in
DataNode$DataXceiver
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533533 ]
Hairong Kuang commented on HADOOP-2018:
---------------------------------------
yes, the bug is fixed in /HADOOP-1912. But since /HADOOP-1912 is not in release 15, I will submit a seprate patch for this.
> Broken pipe SocketException in DataNode$DataXceiver
> ---------------------------------------------------
>
> Key: HADOOP-2018
> URL: https://issues.apache.org/jira/browse/HADOOP-2018
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.0
> Reporter: Konstantin Shvachko
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.15.0
>
>
> I have 2 data-nodes, one of which is trying to replicate blocks to another.
> The second data-node throws the following excpetion for every replicated block.
> {code}
> 07/10/09 20:36:39 INFO dfs.DataNode: Received block blk_-8942388986043611634 from /a.d.d.r:43159
> 07/10/09 20:36:39 WARN dfs.DataNode: Error writing reply back to /a.d.d.r:43159for writing block blk_-8942388986043611634
> 07/10/09 20:36:39 WARN dfs.DataNode: java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
> at java.io.DataOutputStream.writeShort(DataOutputStream.java:151)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:939)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> # It looks like that the first data-node does not expect to receive anything from the second one and closes the connection.
> # There should be a space in front of
> {code}
> + "for writing block " + block );
> {code}
> # The port number is misleading in these messages. DataXceivers open sockets on different ports every time, which is
> different from the data-node's main port. So we should rather print here the main port in order to be able to recognize
> wich data-node the block was sent from.
> Is this related to HADOOP-1908?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2018) Broken pipe SocketException in
DataNode$DataXceiver
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533934 ]
Konstantin Shvachko commented on HADOOP-2018:
---------------------------------------------
The leading space symbol in " for writing block" is still missing.
Other than that the patch works greate.
> Broken pipe SocketException in DataNode$DataXceiver
> ---------------------------------------------------
>
> Key: HADOOP-2018
> URL: https://issues.apache.org/jira/browse/HADOOP-2018
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.0
> Reporter: Konstantin Shvachko
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.15.0
>
> Attachments: pipe.patch
>
>
> I have 2 data-nodes, one of which is trying to replicate blocks to another.
> The second data-node throws the following excpetion for every replicated block.
> {code}
> 07/10/09 20:36:39 INFO dfs.DataNode: Received block blk_-8942388986043611634 from /a.d.d.r:43159
> 07/10/09 20:36:39 WARN dfs.DataNode: Error writing reply back to /a.d.d.r:43159for writing block blk_-8942388986043611634
> 07/10/09 20:36:39 WARN dfs.DataNode: java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
> at java.io.DataOutputStream.writeShort(DataOutputStream.java:151)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:939)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> # It looks like that the first data-node does not expect to receive anything from the second one and closes the connection.
> # There should be a space in front of
> {code}
> + "for writing block " + block );
> {code}
> # The port number is misleading in these messages. DataXceivers open sockets on different ports every time, which is
> different from the data-node's main port. So we should rather print here the main port in order to be able to recognize
> wich data-node the block was sent from.
> Is this related to HADOOP-1908?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2018) Broken pipe SocketException in
DataNode$DataXceiver
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534325 ]
Hudson commented on HADOOP-2018:
--------------------------------
Integrated in Hadoop-Nightly #269 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/269/])
> Broken pipe SocketException in DataNode$DataXceiver
> ---------------------------------------------------
>
> Key: HADOOP-2018
> URL: https://issues.apache.org/jira/browse/HADOOP-2018
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.0
> Reporter: Konstantin Shvachko
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.15.0
>
> Attachments: pipe.patch, pipe1.patch
>
>
> I have 2 data-nodes, one of which is trying to replicate blocks to another.
> The second data-node throws the following excpetion for every replicated block.
> {code}
> 07/10/09 20:36:39 INFO dfs.DataNode: Received block blk_-8942388986043611634 from /a.d.d.r:43159
> 07/10/09 20:36:39 WARN dfs.DataNode: Error writing reply back to /a.d.d.r:43159for writing block blk_-8942388986043611634
> 07/10/09 20:36:39 WARN dfs.DataNode: java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
> at java.io.DataOutputStream.writeShort(DataOutputStream.java:151)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:939)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
> at java.lang.Thread.run(Thread.java:619)
> {code}
> # It looks like that the first data-node does not expect to receive anything from the second one and closes the connection.
> # There should be a space in front of
> {code}
> + "for writing block " + block );
> {code}
> # The port number is misleading in these messages. DataXceivers open sockets on different ports every time, which is
> different from the data-node's main port. So we should rather print here the main port in order to be able to recognize
> wich data-node the block was sent from.
> Is this related to HADOOP-1908?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.