You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Amar Kamat (JIRA)" <ji...@apache.org> on 2008/07/15 12:43:32 UTC
[jira] Commented: (HADOOP-3760) DFS operations fail because of
Stream closed error
[ https://issues.apache.org/jira/browse/HADOOP-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12613576#action_12613576 ]
Amar Kamat commented on HADOOP-3760:
------------------------------------
Here is the complete error
{code}
08/07/15 10:21:04 INFO mapred.FileInputFormat: Total input paths to process : 100
java.io.IOException: Stream closed.
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.isClosed(DFSClient.java:2240)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:2744)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:2652)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:59)
at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:79)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:726)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:966)
at org.apache.hadoop.examples.Sort.run(Sort.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.examples.Sort.main(Sort.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
Exception closing file system.dir/job/job.xml
java.io.IOException: Stream closed.
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.isClosed(DFSClient.java:2240)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:2687)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:2652)
at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:220)
at org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:236)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1379)
at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:230)
at org.apache.hadoop.fs.FileSystem$ClientFinalizer.run(FileSystem.java:215)
{code}
The way this can be reproduced is
1) Run random-writer
2) Run sort ... here the job submission fails with the above mentioned error.
Also reduce tasks fail with the error
{code}
2008-07-15 09:31:28,631 INFO org.apache.hadoop.hdfs.DFSClient: org.apache.hadoop.ipc.RemoteException:
org.apache.hadoop.hdfs.server.namenode.NotReplicatedYetException: Not replicated yet:tmp-dir/part-00000
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1115)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:340)
at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
at org.apache.hadoop.ipc.Client.call(Client.java:707)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216)
at $Proxy1.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
at $Proxy1.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2445)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2328)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1800(DFSClient.java:1740)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1917)
{code}
> DFS operations fail because of Stream closed error
> --------------------------------------------------
>
> Key: HADOOP-3760
> URL: https://issues.apache.org/jira/browse/HADOOP-3760
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Amar Kamat
> Priority: Blocker
>
> DFS operations fail because of {{java.io.IOException: Stream closed.}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.