You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Ravi Phulari (JIRA)" <ji...@apache.org> on 2009/03/14 19:32:50 UTC
[jira] Created: (HADOOP-5492) Quota exceed exception creates file
of size 0
Quota exceed exception creates file of size 0
---------------------------------------------
Key: HADOOP-5492
URL: https://issues.apache.org/jira/browse/HADOOP-5492
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Affects Versions: 0.18.3
Reporter: Ravi Phulari
Empty file of size 0 is created when QuotaExceed exception occurs while copying a file. This file is created with the same name of which file copy is tried .
I.E if operation
Hadoop fs -copyFromLocal testFile1 /testDir
Fails due to quota exceed exception then testFile1 of size 0 is created in testDir on HDFS.
Steps to verify
1) Create testDir and apply space quota of 16kb
2) Copy file say testFile of size greater than 16kb from local file system
3) You should see QuotaException error
4) testFile of size 0 is created in testDir which is not expected .
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5492) Quota exceed exception creates file
of size 0
Posted by "Ravi Phulari (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682085#action_12682085 ]
Ravi Phulari commented on HADOOP-5492:
--------------------------------------
This bug is reproducible .
Stack trace --
localhost:hadoop-0.20.0-030409A rphulari$ bin/hadoop fs -copyFromLocal build.xml testSQ
09/03/13 17:27:02 WARN hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.hdfs.protocol.QuotaExceededException:
org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/rphulari/testSQ is exceeded: namespace
quota=-1 file count=3, diskspace quota=16384 diskspace=67108864
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:96)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:58)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2856)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2736)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2000(DFSClient.java:2036)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2223)
Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of
/user/rphulari/testSQ is exceeded: namespace quota=-1 file count=3, diskspace quota=16384 diskspace=67108864
at org.apache.hadoop.hdfs.server.namenode.INodeDirectoryWithQuota.verifyQuota(INodeDirectoryWithQuota.java:161)
at
org.apache.hadoop.hdfs.server.namenode.INodeDirectoryWithQuota.updateNumItemsInTree(INodeDirectoryWithQuota.java:134)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.updateCount(FSDirectory.java:859)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.addBlock(FSDirectory.java:265)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.allocateBlock(FSNamesystem.java:1419)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1266)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
at org.apache.hadoop.ipc.Client.call(Client.java:730)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
at $Proxy0.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
at $Proxy0.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2854)
... 3 more
09/03/13 17:27:02 WARN hdfs.DFSClient: Error Recovery for block null bad datanode[0] nodes == null
09/03/13 17:27:02 WARN hdfs.DFSClient: Could not get block locations. Source file
"/user/rphulari/testSQ/build.xml" - Aborting...
copyFromLocal: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/rphulari/testSQ is exceeded:
namespace quota=-1 file count=3, diskspace quota=16384 diskspace=67108864
> Quota exceed exception creates file of size 0
> ---------------------------------------------
>
> Key: HADOOP-5492
> URL: https://issues.apache.org/jira/browse/HADOOP-5492
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Ravi Phulari
>
> Empty file of size 0 is created when QuotaExceed exception occurs while copying a file. This file is created with the same name of which file copy is tried .
> I.E if operation
> Hadoop fs -copyFromLocal testFile1 /testDir
> Fails due to quota exceed exception then testFile1 of size 0 is created in testDir on HDFS.
> Steps to verify
> 1) Create testDir and apply space quota of 16kb
> 2) Copy file say testFile of size greater than 16kb from local file system
> 3) You should see QuotaException error
> 4) testFile of size 0 is created in testDir which is not expected .
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5492) Quota exceed exception creates file
of size 0
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682442#action_12682442 ]
Raghu Angadi commented on HADOOP-5492:
--------------------------------------
> Empty file of size 0 is created when QuotaExceed exception occurs while copying a file.
clarification : Neither the exception nor the operation that results in the exception creates a file. The file already exists.
In this case, copyFromLocal does '{{create(file); write(file, data); close(file);}}', and write() operation throws this exception. As expected, the created file exists on HDFS. How copyFromLocal deals with such failed copies (and related files) should really be part of implementation of the shell command.
> Quota exceed exception creates file of size 0
> ---------------------------------------------
>
> Key: HADOOP-5492
> URL: https://issues.apache.org/jira/browse/HADOOP-5492
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Ravi Phulari
>
> Empty file of size 0 is created when QuotaExceed exception occurs while copying a file. This file is created with the same name of which file copy is tried .
> I.E if operation
> Hadoop fs -copyFromLocal testFile1 /testDir
> Fails due to quota exceed exception then testFile1 of size 0 is created in testDir on HDFS.
> Steps to verify
> 1) Create testDir and apply space quota of 16kb
> 2) Copy file say testFile of size greater than 16kb from local file system
> 3) You should see QuotaException error
> 4) testFile of size 0 is created in testDir which is not expected .
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.