You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Neil Joshi (Jira)" <ji...@apache.org> on 2021/03/19 04:27:00 UTC
[jira] [Comment Edited] (HDDS-3752) ozone "fs -ls" cann't list
"o3fs://bucket.volume" without tail "/"
[ https://issues.apache.org/jira/browse/HDDS-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304632#comment-17304632 ]
Neil Joshi edited comment on HDDS-3752 at 3/19/21, 4:26 AM:
------------------------------------------------------------
On the ozone ofs shell cli command for -ls requiring a closing slash to list keys under volume/bucket,
it appears that this is due to how the hadoop fs shell handles paths, specifically the PathData class of org.apache.hadoop.fs.shell which specifies two types of paths:
# Absolute paths, ending with a trailing slash '/'
# Relative paths, no slash and interpreted as relative to 'working directory', default is /user/<username>
Due to a non-trailing slash entry treated as a 'Relative Path', the ozone file system adapter for ofs, osf://, is presented with a path that prefixed with the relative path /user/<username> for which NO key exists thus failing.
_BasicOzoneFileSystem.java, hadoop-ozone-filesystem-common_
{code:java}
@Override
public FileStatus getFileStatus(Path f) throws IOException {
incrementCounter(Statistic.INVOCATION_GET_FILE_STATUS, 1);
statistics.incrementReadOps(1);
LOG.trace("getFileStatus() path:{}", f);
Path qualifiedPath = f.makeQualified(uri, workingDir);
String key = pathToKey(qualifiedPath);{code}
qualifiedPath is relative path.
{code:java}
ls: `o3fs://bucket1.vol1': No such file or directory{code}
was (Author: neiljoshi):
On the ozone ofs shell cli command for -ls requiring a closing slash to list keys under volume/bucket,
it appears that this is due to how the hadoop fs shell handles paths, specifically the PathData class of org.apache.hadoop.fs.shell which specifies two types of paths:
# Absolute paths, ending with a trailing slash '/'
Relative paths, no slash and interpreted as relative to 'working directory', default is /user/<username>
# Due to a non-trailing slash entry treated as a 'Relative Path', the ozone file system adapter for ofs, osf://, is presented with a path that prefixed with the relative path /user/<username> for which NO key exists thus failing.
_BasicOzoneFileSystem.java, hadoop-ozone-filesystem-common_
{code:java}
@Override
public FileStatus getFileStatus(Path f) throws IOException {
incrementCounter(Statistic.INVOCATION_GET_FILE_STATUS, 1);
statistics.incrementReadOps(1);
LOG.trace("getFileStatus() path:{}", f);
Path qualifiedPath = f.makeQualified(uri, workingDir);
String key = pathToKey(qualifiedPath);{code}
qualifiedPath is relative path.
{code:java}
ls: `o3fs://bucket1.vol1': No such file or directory{code}
> ozone "fs -ls" cann't list "o3fs://bucket.volume" without tail "/"
> ------------------------------------------------------------------
>
> Key: HDDS-3752
> URL: https://issues.apache.org/jira/browse/HDDS-3752
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Sammi Chen
> Assignee: Neil Joshi
> Priority: Major
>
> ozone fs -ls o3fs://hive.tdw
> ls: `o3fs://hive.tdw': No such file or directory
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org