You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Runping Qi (JIRA)" <ji...@apache.org> on 2007/10/12 18:50:50 UTC
[jira] Updated: (HADOOP-2042) distcp job failed
[ https://issues.apache.org/jira/browse/HADOOP-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Runping Qi updated HADOOP-2042:
-------------------------------
Component/s: dfs
Description:
I was running distcp to copy data from one dfs to another.
The job failed with the following exception in the mappers:
java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:96)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1633)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:1720)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:49)
at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:64)
at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.copy(CopyFiles.java:305)
at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.map(CopyFiles.java:352)
at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.map(CopyFiles.java:217)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:195)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1750)
I examined the data node logs of the target dfs. I saw a lot of exceptions like:
2007-10-12 15:04:09,109 ERROR org.apache.hadoop.dfs.DataNode: DataXceiver: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at org.apache.hadoop.dfs.DataNode$BlockReceiver.receiveBlock(DataNode.java:1365)
at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:897)
at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
at java.lang.Thread.run(Thread.java:619)
was:
I was running distcp to copy data from one dfs to another.
The job failed with the following exception in the mappers:
java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:96)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1633)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:1720)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:49)
at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:64)
at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.copy(CopyFiles.java:305)
at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.map(CopyFiles.java:352)
at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.map(CopyFiles.java:217)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:195)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1750)
I examined the data node logs of the target dfs. I saw a lot of exceptions like:
2007-10-12 15:04:09,109 ERROR org.apache.hadoop.dfs.DataNode: DataXceiver: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at org.apache.hadoop.dfs.DataNode$BlockReceiver.receiveBlock(DataNode.java:1365)
at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:897)
at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
at java.lang.Thread.run(Thread.java:619)
> distcp job failed
> -----------------
>
> Key: HADOOP-2042
> URL: https://issues.apache.org/jira/browse/HADOOP-2042
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.0
> Reporter: Runping Qi
>
> I was running distcp to copy data from one dfs to another.
> The job failed with the following exception in the mappers:
> java.net.SocketException: Connection reset
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:96)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1633)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:1720)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:49)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:64)
> at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.copy(CopyFiles.java:305)
> at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.map(CopyFiles.java:352)
> at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.map(CopyFiles.java:217)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:195)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1750)
> I examined the data node logs of the target dfs. I saw a lot of exceptions like:
> 2007-10-12 15:04:09,109 ERROR org.apache.hadoop.dfs.DataNode: DataXceiver: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.hadoop.dfs.DataNode$BlockReceiver.receiveBlock(DataNode.java:1365)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:897)
> at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:763)
> at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.