You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Mukund Madhugiri (JIRA)" <ji...@apache.org> on 2007/11/12 19:57:50 UTC
[jira] Created: (HADOOP-2190) dfs ls and lsr commands differ from
POSIX standards
dfs ls and lsr commands differ from POSIX standards
---------------------------------------------------
Key: HADOOP-2190
URL: https://issues.apache.org/jira/browse/HADOOP-2190
Project: Hadoop
Issue Type: Bug
Components: dfs
Affects Versions: 0.15.1, 0.16.0
Reporter: Mukund Madhugiri
Fix For: 0.16.0
Assuming the dfs commands follow POSIX standards, there are some problems with the DFS ls and lsr commands. I compared the DFS output with that of RHEL 4u5
1. ls a directory when there are no files/directories in that directory:
Linux: No output
DFS: Found 0 items
2. ls a file/directory that does not exist:
Linux: ls: /doesnotexist: No such file or directory
DFS: Found 0 items
3. lsr a directory that does not exist:
Linux: ls: /doesnotexist: No such file or directory
DFS: No output
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2190) dfs ls and lsr commands differ from
POSIX standards
Posted by "lohit vijayarenu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
lohit vijayarenu updated HADOOP-2190:
-------------------------------------
Attachment: HADOOP-2190-1.patch
Attached patch throws out error for ls, lsr, du, dus commands for non-existing files. There were 2 changes DistributedFileSyste.listPaths() was not returning null for non-existing files and FileSystem.listPaths() was consuming null returned from underlying listPaths and returning zero length Path[] array. So, I have made changes at few places so that listPaths returns null when required. I ran the tests and they look good. If anyone could take a look at the changes and approve it, I will prepare test cases and update another patch. The issue reported was also seen on local filesystem. fixing the second issue also takes care of local filesystem.
Below is the output after the patch
{noformat}
ls/lsr command
[ hadoop-trunk]$ hadoop dfs -ls empty
Found 0 items
[ hadoop-trunk]$ hadoop dfs -ls nofile
ls: Could not get listing for nofile
[ hadoop-trunk]$ hadoop dfs -lsr nofile
lsr: Could not get listing for nofile
du command
[ hadoop-trunk]$ hadoop dfs -du empty
Found 0 items
[ hadoop-trunk]$ hadoop dfs -du nofile
du: Could not get listing for nofile
[ hadoop-trunk]$ hadoop dfs -dus empty
empty 0
[ hadoop-trunk]$ hadoop dfs -dus nofile
dus: dus: No match: nofile
[ hadoop-trunk]$
{noformat}
> dfs ls and lsr commands differ from POSIX standards
> ---------------------------------------------------
>
> Key: HADOOP-2190
> URL: https://issues.apache.org/jira/browse/HADOOP-2190
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.1, 0.16.0
> Reporter: Mukund Madhugiri
> Fix For: 0.16.0
>
> Attachments: HADOOP-2190-1.patch
>
>
> Assuming the dfs commands follow POSIX standards, there are some problems with the DFS ls and lsr commands. I compared the DFS output with that of RHEL 4u5
> 1. ls a directory when there are no files/directories in that directory:
> Linux: No output
> DFS: Found 0 items
> 2. ls a file/directory that does not exist:
> Linux: ls: /doesnotexist: No such file or directory
> DFS: Found 0 items
> 3. lsr a directory that does not exist:
> Linux: ls: /doesnotexist: No such file or directory
> DFS: No output
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2190) dfs ls and lsr commands differ from
POSIX standards
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12543221 ]
dhruba borthakur commented on HADOOP-2190:
------------------------------------------
It will be a good idea to remove the dependency of this patch on the deprecated FileSystem.listPath API
> dfs ls and lsr commands differ from POSIX standards
> ---------------------------------------------------
>
> Key: HADOOP-2190
> URL: https://issues.apache.org/jira/browse/HADOOP-2190
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.1, 0.16.0
> Reporter: Mukund Madhugiri
> Fix For: 0.16.0
>
> Attachments: HADOOP-2190-1.patch
>
>
> Assuming the dfs commands follow POSIX standards, there are some problems with the DFS ls and lsr commands. I compared the DFS output with that of RHEL 4u5
> 1. ls a directory when there are no files/directories in that directory:
> Linux: No output
> DFS: Found 0 items
> 2. ls a file/directory that does not exist:
> Linux: ls: /doesnotexist: No such file or directory
> DFS: Found 0 items
> 3. lsr a directory that does not exist:
> Linux: ls: /doesnotexist: No such file or directory
> DFS: No output
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2190) dfs ls and lsr commands differ from
POSIX standards
Posted by "Mukund Madhugiri (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12542930 ]
Mukund Madhugiri commented on HADOOP-2190:
------------------------------------------
Can the command output be along the lines of what we see on linux?
RHEL throws this:
ls: /doesnotexist: No such file or directory
> dfs ls and lsr commands differ from POSIX standards
> ---------------------------------------------------
>
> Key: HADOOP-2190
> URL: https://issues.apache.org/jira/browse/HADOOP-2190
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.1, 0.16.0
> Reporter: Mukund Madhugiri
> Fix For: 0.16.0
>
> Attachments: HADOOP-2190-1.patch
>
>
> Assuming the dfs commands follow POSIX standards, there are some problems with the DFS ls and lsr commands. I compared the DFS output with that of RHEL 4u5
> 1. ls a directory when there are no files/directories in that directory:
> Linux: No output
> DFS: Found 0 items
> 2. ls a file/directory that does not exist:
> Linux: ls: /doesnotexist: No such file or directory
> DFS: Found 0 items
> 3. lsr a directory that does not exist:
> Linux: ls: /doesnotexist: No such file or directory
> DFS: No output
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2190) dfs ls and lsr commands differ from
POSIX standards
Posted by "lohit vijayarenu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12542979 ]
lohit vijayarenu commented on HADOOP-2190:
------------------------------------------
sure, if patch is approved I will make these changes while adding testcase.
> dfs ls and lsr commands differ from POSIX standards
> ---------------------------------------------------
>
> Key: HADOOP-2190
> URL: https://issues.apache.org/jira/browse/HADOOP-2190
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.1, 0.16.0
> Reporter: Mukund Madhugiri
> Fix For: 0.16.0
>
> Attachments: HADOOP-2190-1.patch
>
>
> Assuming the dfs commands follow POSIX standards, there are some problems with the DFS ls and lsr commands. I compared the DFS output with that of RHEL 4u5
> 1. ls a directory when there are no files/directories in that directory:
> Linux: No output
> DFS: Found 0 items
> 2. ls a file/directory that does not exist:
> Linux: ls: /doesnotexist: No such file or directory
> DFS: Found 0 items
> 3. lsr a directory that does not exist:
> Linux: ls: /doesnotexist: No such file or directory
> DFS: No output
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2190) dfs ls and lsr commands differ from
POSIX standards
Posted by "lohit vijayarenu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
lohit vijayarenu updated HADOOP-2190:
-------------------------------------
Attachment: HADOOP-2190-2.patch
Another patch, which now calls listStatus instead of deprecated listPaths(Path) . updated test case and changes suggested by Mukund.
> dfs ls and lsr commands differ from POSIX standards
> ---------------------------------------------------
>
> Key: HADOOP-2190
> URL: https://issues.apache.org/jira/browse/HADOOP-2190
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.1, 0.16.0
> Reporter: Mukund Madhugiri
> Fix For: 0.16.0
>
> Attachments: HADOOP-2190-1.patch, HADOOP-2190-2.patch
>
>
> Assuming the dfs commands follow POSIX standards, there are some problems with the DFS ls and lsr commands. I compared the DFS output with that of RHEL 4u5
> 1. ls a directory when there are no files/directories in that directory:
> Linux: No output
> DFS: Found 0 items
> 2. ls a file/directory that does not exist:
> Linux: ls: /doesnotexist: No such file or directory
> DFS: Found 0 items
> 3. lsr a directory that does not exist:
> Linux: ls: /doesnotexist: No such file or directory
> DFS: No output
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.