You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Raghu Angadi (JIRA)" <ji...@apache.org> on 2008/07/01 22:44:45 UTC
[jira] Created: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
--------------------------------------------------------------------------------------------------
Key: HADOOP-3678
URL: https://issues.apache.org/jira/browse/HADOOP-3678
Project: Hadoop Core
Issue Type: Bug
Affects Versions: 0.17.0
Reporter: Raghu Angadi
Assignee: Raghu Angadi
When a client reads data using read(), it closes the sockets after it is done. Often it might read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed of end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12673942#action_12673942 ]
Hudson commented on HADOOP-3678:
--------------------------------
Integrated in Hadoop-trunk #756 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/756/])
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.2
>
> Attachments: HADOOP-3678-branch-17.patch, HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Rong-En Fan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12610122#action_12610122 ]
Rong-En Fan commented on HADOOP-3678:
-------------------------------------
Can we backport this into 0.17, too?
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
> Attachments: HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3678:
---------------------------------
Description: When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user. (was:
When a client reads data using read(), it closes the sockets after it is done. Often it might read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed of end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.)
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12610050#action_12610050 ]
Raghu Angadi commented on HADOOP-3678:
--------------------------------------
Regd unit tests : This is a minor fix to avoid a spurious error. There is no functionality change.
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
> Attachments: HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3678:
---------------------------------
Attachment: HADOOP-3678.patch
Suggested patch. It does a minor code reorg in sendChunks().
Note that while using transferTo(), we can not easily distinguish if the error occurred while reading from the disk or while writing to the client, though most likely its while writing to the client. One hack is to check the exception message, but that is usually not encouraged.
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
> Attachments: HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Rong-En Fan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614204#action_12614204 ]
Rong-En Fan commented on HADOOP-3678:
-------------------------------------
can we fix this in 0.17.2, too?
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
> Attachments: HADOOP-3678-branch-17.patch, HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3678:
---------------------------------
Status: Patch Available (was: Open)
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
> Attachments: HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12610037#action_12610037 ]
Hadoop QA commented on HADOOP-3678:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12385136/HADOOP-3678.patch
against trunk revision 673445.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2783/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2783/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2783/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2783/console
This message is automatically generated.
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
> Attachments: HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655311#action_12655311 ]
Raghu Angadi commented on HADOOP-3678:
--------------------------------------
Aah.. missed the last 4 bytes after sending the block data. will have a fix. I don't think this should be a blocker but better to be in 0.18.3.
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.2
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.2
>
> Attachments: HADOOP-3678-branch-17.patch, HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi reopened HADOOP-3678:
----------------------------------
Reopening it to recommend it for 0.17.2.
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.17.2, 0.18.0
>
> Attachments: HADOOP-3678-branch-17.patch, HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3678:
---------------------------------
Fix Version/s: 0.18.0
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Cosmin Lehene (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655173#action_12655173 ]
Cosmin Lehene commented on HADOOP-3678:
---------------------------------------
I'm getting this with Hadoop 0.18.2. Reopening
2008-12-10 02:00:17,059 WARN org.apache.hadoop.dfs.DataNode: DatanodeRegistration(10.72.7.153:50010, storageID=DS-594776053-127.0.0.1-50010-1223411294140, infoPort=50075, ipcPort=50020):Got exception w
hile serving blk_698267671609892267_112619 to /10.72.7.153:
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:104)
at sun.nio.ch.IOUtil.write(IOUtil.java:75)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:55)
at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:140)
at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:146)
at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:107)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at org.apache.hadoop.dfs.DataNode$BlockSender.sendBlock(DataNode.java:2019)
at org.apache.hadoop.dfs.DataNode$DataXceiver.readBlock(DataNode.java:1140)
at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:1068)
at java.lang.Thread.run(Thread.java:619)
2008-12-10 02:00:17,060 ERROR org.apache.hadoop.dfs.DataNode: DatanodeRegistration(10.72.7.153:50010, storageID=DS-594776053-127.0.0.1-50010-1223411294140, infoPort=50075, ipcPort=50020):DataXceiver: j
ava.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:104)
at sun.nio.ch.IOUtil.write(IOUtil.java:75)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:55)
at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:140)
at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:146)
at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:107)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at org.apache.hadoop.dfs.DataNode$BlockSender.sendBlock(DataNode.java:2019)
at org.apache.hadoop.dfs.DataNode$DataXceiver.readBlock(DataNode.java:1140)
at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:1068)
at java.lang.Thread.run(Thread.java:619)
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.2
>
> Attachments: HADOOP-3678-branch-17.patch, HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614213#action_12614213 ]
Raghu Angadi commented on HADOOP-3678:
--------------------------------------
+1 for 0.17.2.
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
> Attachments: HADOOP-3678-branch-17.patch, HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "LN (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12610179#action_12610179 ]
LN commented on HADOOP-3678:
----------------------------
need a patch for 0.17.1 release too.
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
> Attachments: HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3678:
---------------------------------
Resolution: Fixed
Release Note: Avoid spurious exceptions logged at DataNode when clients read from DFS.
Status: Resolved (was: Patch Available)
I just committed this.
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
> Attachments: HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3678:
---------------------------------
Attachment: HADOOP-3678-branch-17.patch
Patch for 0.17 attached.
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
> Attachments: HADOOP-3678-branch-17.patch, HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12610003#action_12610003 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3678:
------------------------------------------------
Patch looks good. Some minor comments:
- It is better to compare Class object instead of String.
- Could you add some message saying that the new SocketException is caused by the IOException?
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
> Attachments: HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3678:
---------------------------------
Affects Version/s: (was: 0.18.2)
0.17.0
Cosmin, I filed HADOOP-4862 to fix the log you are seeing.
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.2
>
> Attachments: HADOOP-3678-branch-17.patch, HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3678:
---------------------------------
Attachment: HADOOP-3678.patch
Thats better. Thanks Nicholas.
Attached patch incorporates the changes.
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
> Attachments: HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12610275#action_12610275 ]
Raghu Angadi commented on HADOOP-3678:
--------------------------------------
I will attach a patch for 0.17.
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
> Attachments: HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624733#action_12624733 ]
Hudson commented on HADOOP-3678:
--------------------------------
Integrated in Hadoop-trunk #581 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/581/])
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.2
>
> Attachments: HADOOP-3678-branch-17.patch, HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12610213#action_12610213 ]
Hudson commented on HADOOP-3678:
--------------------------------
Integrated in Hadoop-trunk #537 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/537/])
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
> Attachments: HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3678:
---------------------------------
Priority: Blocker (was: Major)
Fix Version/s: 0.17.2
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.2, 0.18.0
>
> Attachments: HADOOP-3678-branch-17.patch, HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Cosmin Lehene (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cosmin Lehene updated HADOOP-3678:
----------------------------------
Affects Version/s: (was: 0.17.0)
0.18.2
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.2
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.2
>
> Attachments: HADOOP-3678-branch-17.patch, HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3678:
-------------------------------------------
Hadoop Flags: [Reviewed]
+1
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.18.0
>
> Attachments: HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-3678) Avoid spurious "DataXceiver:
java.io.IOException: Connection reset by peer" errors in DataNode log
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi resolved HADOOP-3678.
----------------------------------
Resolution: Fixed
I just committed this to 0.17.2 and moved CHANGES.txt entry on the other lines.
> Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3678
> URL: https://issues.apache.org/jira/browse/HADOOP-3678
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.2, 0.18.0
>
> Attachments: HADOOP-3678-branch-17.patch, HADOOP-3678.patch, HADOOP-3678.patch
>
>
> When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.