You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Koji Noguchi (JIRA)" <ji...@apache.org> on 2008/03/31 19:54:24 UTC
[jira] Created: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
distcp fail copying to /user/<username>/<newtarget> (with permission on)
------------------------------------------------------------------------
Key: HADOOP-3138
URL: https://issues.apache.org/jira/browse/HADOOP-3138
Project: Hadoop Core
Issue Type: Bug
Components: util
Affects Versions: 0.16.1
Reporter: Koji Noguchi
When distcp-ing to /user/<username>/<newtarget>, I get an error with
Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
{noformat}
at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
at org.apache.hadoop.ipc.Client.call(Client.java:512)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
{noformat}
In distcp set up, we have
{noformat}
if (!dstExists || !dstIsDir) {
Path parent = destPath.getParent();
dstfs.mkdirs(parent);
logPath = new Path(parent, filename);
}
{noformat}
We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585973#action_12585973 ]
Hudson commented on HADOOP-3138:
--------------------------------
Integrated in Hadoop-trunk #451 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/451/])
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch, HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3138:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this.
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch, HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-3138:
------------------------------------
Fix Version/s: 0.17.0
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: util
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.17.0
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3138:
---------------------------------
Attachment: HADOOP-3138.patch
mkdirs() first invokes {{exists()}} at the beginning and returns false if the path exists. yes, exists() is called even before the check for safemode().
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi reassigned HADOOP-3138:
------------------------------------
Assignee: Raghu Angadi (was: Tsz Wo (Nicholas), SZE)
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: util
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585332#action_12585332 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3138:
------------------------------------------------
> ... , which is wrong.
+1 I agree that the semantic of mkdirs(...) is not clear. This should be fixed in HADOOP-3163.
Not sure we still need to fix this issue since the distcp problem is already gone by HADOOP-3099. Should we mark this one as WON'T FIX and then work on HADOOP-3163?
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch, HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585400#action_12585400 ]
Hadoop QA commented on HADOOP-3138:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12379316/HADOOP-3138.patch
against trunk revision 643282.
@author +1. The patch does not contain any @author tags.
tests included +1. The patch appears to include 3 new or modified tests.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2152/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2152/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2152/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2152/console
This message is automatically generated.
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch, HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585257#action_12585257 ]
Hadoop QA commented on HADOOP-3138:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12379230/HADOOP-3138.patch
against trunk revision 643282.
@author +1. The patch does not contain any @author tags.
tests included +1. The patch appears to include 3 new or modified tests.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests -1. The patch failed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2142/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2142/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2142/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2142/console
This message is automatically generated.
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-3138:
------------------------------------
Priority: Blocker (was: Major)
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: util
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.17.0
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3138:
---------------------------------
Component/s: (was: util)
dfs
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585334#action_12585334 ]
rangadi edited comment on HADOOP-3138 at 4/3/08 3:44 PM:
--------------------------------------------------------------
Since this is a bug in DFS, it is likely to affect other users as well.. my preference would be to commit it.
was (Author: rangadi):
Since this is a bug in DFS, it is likely affect other users as well.. my preference would be to commit it.
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch, HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585855#action_12585855 ]
Konstantin Shvachko commented on HADOOP-3138:
---------------------------------------------
+1. The patch is good for 0.16.3
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch, HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3138:
---------------------------------
Status: Patch Available (was: Open)
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch, HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Mukund Madhugiri (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mukund Madhugiri updated HADOOP-3138:
-------------------------------------
Fix Version/s: (was: 0.17.0)
0.16.4
I committed this to 0.16.4. Thanks Raghu.
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.16.4
>
> Attachments: HADOOP-3138.patch, HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585279#action_12585279 ]
Raghu Angadi commented on HADOOP-3138:
--------------------------------------
hmm... in multiple locations, there is code like {{' if (!mkdir(dir)) { throw new IOException("could not create dir"); } '}} , which is wrong. I can fix the once that show up on the unit test. Alternately just we could continue to allow this kind of use...
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12583830#action_12583830 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3138:
------------------------------------------------
> Should this be a client side fix?
Tried the following in Linux:
- mkdir(const char *pathname, mode_t mode) system call. It returns " pathname already exists".
- For mkdir -p, it does not report any error if the path exists and the path is a directory, even if there is no w permission in the parent.
- For mkdir, it reports "File exists" error.
I guess Raghu is right since our mkdirs and mkdir -p have similar semantic.
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: util
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Tsz Wo (Nicholas), SZE
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12583782#action_12583782 ]
Raghu Angadi commented on HADOOP-3138:
--------------------------------------
> We should check if parent path exists before calling mkdir?
Should this be a client side fix? HDFS mkdirs is is like 'mkdir -p', if the dir exists it should pass (at least according my Linux machine anyway: {{`mkdir -p /usr`}} succeeds without error though I don't have permission to create /usr). So it looks like if a directory exists, then creating that directory should not result in a access denied (unless the user does not have directory-read permissions to the parent directory, may be).
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: util
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Tsz Wo (Nicholas), SZE
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3138:
---------------------------------
Status: Patch Available (was: Open)
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585279#action_12585279 ]
rangadi edited comment on HADOOP-3138 at 4/3/08 1:52 PM:
--------------------------------------------------------------
hmm... in multiple locations, there is code like {{' if (!mkdir(dir)) { throw new IOException("could not create dir"); } '}} , which is wrong. I can fix the ones that show up on the unit test. Alternately we could just continue to allow this kind of use...
was (Author: rangadi):
hmm... in multiple locations, there is code like {{' if (!mkdir(dir)) { throw new IOException("could not create dir"); } '}} , which is wrong. I can fix the once that show up on the unit test. Alternately just we could continue to allow this kind of use...
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3138:
---------------------------------
Status: Open (was: Patch Available)
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3138:
---------------------------------
Attachment: HADOOP-3138.patch
The next patch attached continue to enforce the assumption on mkdirs() return value.
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch, HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585334#action_12585334 ]
Raghu Angadi commented on HADOOP-3138:
--------------------------------------
Since this is a bug in DFS, it is likely affect other users as well.. my preference would be to commit it.
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch, HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE reassigned HADOOP-3138:
----------------------------------------------
Assignee: Tsz Wo (Nicholas), SZE
> We should check if parent path exists before calling mkdir?
Sure. I will fix it.
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: util
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Tsz Wo (Nicholas), SZE
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3138) distcp fail copying to
/user// (with permission on)
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3138:
-------------------------------------------
Hadoop Flags: [Reviewed]
+1 patch looks good
> distcp fail copying to /user/<username>/<newtarget> (with permission on)
> ------------------------------------------------------------------------
>
> Key: HADOOP-3138
> URL: https://issues.apache.org/jira/browse/HADOOP-3138
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.1
> Reporter: Koji Noguchi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3138.patch
>
>
> When distcp-ing to /user/<username>/<newtarget>, I get an error with
> Copy failed: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=knoguchi, access=WRITE, inode="user":superuser:superusergroup:rwxr-xr-x
> {noformat}
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
> at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:154)
> at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:102)
> at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4037)
> at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4007)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1576)
> at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1559)
> at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:422)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> at org.apache.hadoop.ipc.Client.call(Client.java:512)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:550)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:184)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:980)
> at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:735)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:525)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)
> {noformat}
> In distcp set up, we have
> {noformat}
> if (!dstExists || !dstIsDir) {
> Path parent = destPath.getParent();
> dstfs.mkdirs(parent);
> logPath = new Path(parent, filename);
> }
> {noformat}
> We should check if parent path exists before calling mkdir?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.