You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by "Shouheng Yi (JIRA)" <ji...@apache.org> on 2017/07/13 20:09:00 UTC
[jira] [Updated] (REEF-1827) new Uri(path) in REEF.NET IO Returns
Lowercase String
[ https://issues.apache.org/jira/browse/REEF-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shouheng Yi updated REEF-1827:
------------------------------
Description:
In the [Org.Apache.REEF.IO.FileSystem.IFileSystem|https://github.com/apache/reef/blob/3531944f11cc115b105a6bec4268713766425034/lang/cs/Org.Apache.REEF.IO/FileSystem/IFileSystem.cs], we have many methods that are based on C#'s Uri. For example:
{code:none}
void CopyToLocal(Uri remoteFileUri, string localName);
void CreateDirectory(Uri directoryUri);
...
{code}
From the logging and my experiments, this converts path to lowercase characters. For example:
{code:none}
new Uri("hdfs://XYZ123/mypath/data.txt") // Returns "hdfs://xyz123/mypath/data.txt"
{code}
If we are interacting with hdfs, in the subsequent dfs command in [Org.Apache.REEF.IO.FileSystem.Hadoop|https://github.com/apache/reef/blob/3531944f11cc115b105a6bec4268713766425034/lang/cs/Org.Apache.REEF.IO/FileSystem/Hadoop/HadoopFileSystem.cs]:
{code:none}
public void CopyToLocal(Uri remoteFileUri, string localName)
{
_commandRunner.Run("dfs -get " + remoteFileUri + " " + localName);
}
{code}
this will fail, since hdfs is case-sensitive. I'm wondering if we are open to the idea of having some string based APIs that ensure users that they are passing the exact path as they specified in their code. Thanks!
Here is the link to the PR: [#1329|https://github.com/apache/reef/pull/1329]
was:
In the [Org.Apache.REEF.IO.FileSystem.IFileSystem|https://github.com/apache/reef/blob/3531944f11cc115b105a6bec4268713766425034/lang/cs/Org.Apache.REEF.IO/FileSystem/IFileSystem.cs], we have many methods that are based on C#'s Uri. For example:
{code:none}
void CopyToLocal(Uri remoteFileUri, string localName);
void CreateDirectory(Uri directoryUri);
...
{code}
From the logging and my experiments, this converts path to lowercase characters. For example:
{code:none}
new Uri("hdfs://XYZ123/mypath/data.txt") // Returns "hdfs://xyz123/mypath/data.txt"
{code}
If we are interacting with hdfs, in the subsequent dfs command in [Org.Apache.REEF.IO.FileSystem.Hadoop|https://github.com/apache/reef/blob/3531944f11cc115b105a6bec4268713766425034/lang/cs/Org.Apache.REEF.IO/FileSystem/Hadoop/HadoopFileSystem.cs]:
{code:none}
public void CopyToLocal(Uri remoteFileUri, string localName)
{
_commandRunner.Run("dfs -get " + remoteFileUri + " " + localName);
}
{code}
this will fail, since hdfs is case-sensitive. I'm wondering if we are open to the idea of having some string based APIs that ensure users that they are passing the exact path as they specified in their code. Thanks!
> new Uri(path) in REEF.NET IO Returns Lowercase String
> -----------------------------------------------------
>
> Key: REEF-1827
> URL: https://issues.apache.org/jira/browse/REEF-1827
> Project: REEF
> Issue Type: Bug
> Components: REEF.NET IO
> Affects Versions: 0.15
> Reporter: Shouheng Yi
> Priority: Minor
> Labels: security
>
> In the [Org.Apache.REEF.IO.FileSystem.IFileSystem|https://github.com/apache/reef/blob/3531944f11cc115b105a6bec4268713766425034/lang/cs/Org.Apache.REEF.IO/FileSystem/IFileSystem.cs], we have many methods that are based on C#'s Uri. For example:
> {code:none}
> void CopyToLocal(Uri remoteFileUri, string localName);
> void CreateDirectory(Uri directoryUri);
> ...
> {code}
> From the logging and my experiments, this converts path to lowercase characters. For example:
> {code:none}
> new Uri("hdfs://XYZ123/mypath/data.txt") // Returns "hdfs://xyz123/mypath/data.txt"
> {code}
> If we are interacting with hdfs, in the subsequent dfs command in [Org.Apache.REEF.IO.FileSystem.Hadoop|https://github.com/apache/reef/blob/3531944f11cc115b105a6bec4268713766425034/lang/cs/Org.Apache.REEF.IO/FileSystem/Hadoop/HadoopFileSystem.cs]:
> {code:none}
> public void CopyToLocal(Uri remoteFileUri, string localName)
> {
> _commandRunner.Run("dfs -get " + remoteFileUri + " " + localName);
> }
> {code}
> this will fail, since hdfs is case-sensitive. I'm wondering if we are open to the idea of having some string based APIs that ensure users that they are passing the exact path as they specified in their code. Thanks!
> Here is the link to the PR: [#1329|https://github.com/apache/reef/pull/1329]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)