You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Rajit Saha (Created) (JIRA)" <ji...@apache.org> on 2011/10/14 23:30:11 UTC
[jira] [Created] (HDFS-2450) Only complete hostname is supported to
access data via hdfs://
Only complete hostname is supported to access data via hdfs://
--------------------------------------------------------------
Key: HDFS-2450
URL: https://issues.apache.org/jira/browse/HDFS-2450
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.20.205.0
Reporter: Rajit Saha
If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
I am running following in .20.205 Client to get data from .20.205 NN (host1)
$hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
$hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
$hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
Only following is supported
$hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2450) Only complete hostname is supported to
access data via hdfs://
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HDFS-2450:
------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450-3.patch, HDFS-2450-4.patch, HDFS-2450-5.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136030#comment-13136030 ]
Daryn Sharp commented on HDFS-2450:
-----------------------------------
Full test suite passes.
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137524#comment-13137524 ]
Hadoop QA commented on HDFS-2450:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501154/HDFS-2450-4.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1474//console
This message is automatically generated.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450-3.patch, HDFS-2450-4.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130131#comment-13130131 ]
Hadoop QA commented on HDFS-2450:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12499612/HDFS-2450.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1389//console
This message is automatically generated.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450.patch
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Daryn Sharp (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp reassigned HDFS-2450:
---------------------------------
Assignee: Daryn Sharp
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136305#comment-13136305 ]
Daryn Sharp commented on HDFS-2450:
-----------------------------------
# "can you get thatScheme before making if ..."
#* Gladly! That was one of the aforementioned things I was "itching" to do, but removed before posting the patch to minimize the change.
# "All checks seem to be using defaultPort"
#* No, the default port is only used if there is not a port in the uri
# "Why remove?"
#* Default uri is sufficient because {{FileSystem.get(defaultUri, conf).getUri()}} equals {{defaultUri}}. Also note about instantiating the default fs:
#*# It's relatively expensive, and horrifically bad if the fs cache is disabled to create a new fs
#*# If the default fs is down, the thread blocks while rpc retries occur
#*# The thrown {{RuntimeException}} is likely blow up the client, as opposed to the {{IllegalArgumentException}} the caller expects.
# "If thisAuthority and thatAuthority is null, the new mehod throw IllegalArgumentException"
#* No, the == on the authorities will catch this case.
# "If thisAuthority and thatAuthority are not null, you end up parsing the URI for thatAuthority twice."
#* Yes, once before canonicalization, once after canonicalization.
# "DistributedFileSystem#makeQualified remove makes me concerned"
#* The only reason it existed is because {{NameNode.getUri()}} was stripping off the default port. This alone would cause {{p = new Path("hdfs://host:default-port/file"); fs = p.getFileSystem(conf)}} to be incompatible because the path has a port, but the fs stripped it off. So an override of {{checkPath}} and {{makeQualified}} were added to strip the default port from paths too. All of it is completely unnecessary, especially with canonicalization.
Tests:
# full test suite
# manual testing with FsShell "ls":
#* scheme://short
#* scheme://short:port
#* scheme://short.partial
#* scheme://short.partial:port
#* scheme://fqdn
#* scheme://fqdn:port
#* scheme://ip
#* scheme://ip:port
# adding more unit tests to automate stressing {{checkPath}} via {{DFS}}, both positive and negative cases
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13131913#comment-13131913 ]
Daryn Sharp commented on HDFS-2450:
-----------------------------------
Trimmed out as much as I could, reverted the refactor of {{checkPath}}, and removed extraneous comments based on input from Suresh & Nicholas.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450.patch
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2450) Only complete hostname is supported to
access data via hdfs://
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HDFS-2450:
------------------------------
Attachment: HDFS-2450-2.patch
Optimized {{NetUtils.getCanonicalUri()}} to return same uri in majority of cases. DNS canonical resolutions are cached to accelerate the method when use_ip=false.
Optimized {{FileSystem.checkPath()}} to avoid redundant code. I tried to keep it as similar as possible to simplify review. Although I was itching to change it to stop creating locals that won't be used in some cases, or that will be used only once, etc.
{{DistributedFileSystem}} manages its uri like the other filesystems. By not manipulating the given uri, the removed methods are not needed to compensate for the previous manipulations.
Tests appear to be passing. Running full test suite overnight.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2450) Only complete hostname is supported to
access data via hdfs://
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HDFS-2450:
------------------------------
Attachment: IP vs. Hostname.pdf
Attaching a doc to clarify how this jira and 7510 interact. Will revise patch and repost per comments.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127965#comment-13127965 ]
Daryn Sharp commented on HDFS-2450:
-----------------------------------
The exception originates in {{FileSystem.checkPath()}} which is used to see if a {{Path}} belongs to the {{FileSystem}}. Among other things, it does a simple string comparisons of the host in the uri's authority. There is an interaction when setting use_ip=false to enable the use of host-based tokens. The {{NetUtils}} methods to create an {{InetSocketAddress}} will return FQDNs to increase the security of host-based tokens. With a FQDN, there is no ambiguity, regardless of the host's dns search path.
The observed issue is that {{DistributedFileSystem}} does not store its designated uri as-is. Instead it takes the uri's authority, converts it to a {{InetSocketAddress}}, and then converts it back to a string. With use_ip=true, the default behavior, the conversion are unnecessary but always yield the original value. With use_ip=false, the host will become a FQDN after the conversions, so simple string comparison between the FQDN and short hostname fails...
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137124#comment-13137124 ]
Daryn Sharp commented on HDFS-2450:
-----------------------------------
Thanks for the review comments, Suresh!
# I see your point. I had originally thought about adding them there, but reconsidered for the following reasons. If you have a strong preference, I will try to move the tests to {{TestFileSystem}}.
#* I would have to copy-n-paste the test resolver into {{TestFileSystem}}
#* I couldn't use the test resolver from {{TestFileSystem}} w/o increasing the visibility of {{NetUtils}} methods to control the resolver -- which I don't think either of us would like to be publicly visible.
#* While the {{FileSystem}} related tests are exercising the ability of {{checkPath(...)}} to compare canonicals, it also exercising the canonicalizing code in {{NetUtils}}. I admit it's a tenuous connection.
# I contemplated whether {{getCanonicalUri()}} should be public. My final decision was influenced by testability, but the main reason is I realized that external callers cannot directly use {{NetUtils.getCanonicalUri(...)}} since {{FileSystem#getDefaultPort()}} is a protected method. It seems natural to publicly expose {{getUri()}} and {{getCanonicalUri()}}, but if you disagree, I will try to change it.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450-3.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Suresh Srinivas (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132866#comment-13132866 ]
Suresh Srinivas commented on HDFS-2450:
---------------------------------------
# FileSystem.java
#* Please cache canonical URI for defaultURI and FileSystem URI, instead of getting it call to checkPath.
#* Please add more details to getCanonicalUri() method as to what is expected in uri's hostname (FQDN).
#* Code duplication in the following snippet can be avoided by creating a protected static method.
{noformat}
+ if (thisScheme.equalsIgnoreCase(defaultUri.getScheme())) {
+ if (thisAuthority.equalsIgnoreCase(defaultUri.getAuthority()))
+ return;
+ defaultUri = NetUtils.getCanonicalUri(defaultUri, getDefaultPort());
+ thisAuthority = getCanonicalUri().getAuthority();
+ if (thisAuthority.equalsIgnoreCase(defaultUri.getAuthority()))
+ return;
{noformat}
# DistributedFileSystem.java - The existing methods checkPath() and makeQualified() allowed default ports exclusively (indicated in the javadoc). I am not sure why that was done. Why is this patch removing that?
# In tests how does host become host.a.b?
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450.patch
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2450) Only complete hostname is supported to
access data via hdfs://
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HDFS-2450:
------------------------------
Status: Patch Available (was: Open)
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450.patch
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Robert Joseph Evans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135584#comment-13135584 ]
Robert Joseph Evans commented on HDFS-2450:
-------------------------------------------
I am not an expert in HDFS, but I have read through the changes and they look good to me. Only one new public API was added NetUtils.getCanonicalUri, which has tests for it. I am not positive how DistibutedFileSystem's uri is used in all cases, but from what I have read so far it looks OK to not to the port modifications that it was doing previous to this patch.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272441#comment-13272441 ]
Harsh J commented on HDFS-2450:
-------------------------------
Hey Daryn,
I've not verified this by self but does trunk remain unaffected by this issue?
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Fix For: 1.0.0
>
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450-3.patch, HDFS-2450-4.patch, HDFS-2450-5.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272690#comment-13272690 ]
Daryn Sharp commented on HDFS-2450:
-----------------------------------
I'll investigate.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Fix For: 1.0.0
>
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450-3.patch, HDFS-2450-4.patch, HDFS-2450-5.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273284#comment-13273284 ]
Daryn Sharp commented on HDFS-2450:
-----------------------------------
The actual code was brought into trunk quite a while back via porting of 205 dfs/hftp changes since there was a dependency. However, the slew of tests added in 205 were not carried over. I'll post a patch containing the tests.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Fix For: 1.0.0
>
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450-3.patch, HDFS-2450-4.patch, HDFS-2450-5.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2450) Only complete hostname is supported to
access data via hdfs://
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HDFS-2450:
------------------------------
Attachment: HDFS-2450-1.patch
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450.patch
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136713#comment-13136713 ]
Hadoop QA commented on HDFS-2450:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12500997/HDFS-2450-3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1470//console
This message is automatically generated.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450-3.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Suresh Srinivas (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136207#comment-13136207 ]
Suresh Srinivas commented on HDFS-2450:
---------------------------------------
Comments:
# FileSystem#checkPath
#* Unrelated this patch - can you get thatScheme before making {{if (uri.getScheme() == null}} check. This avoids getting scheme twice from the URI.
#* Earlier the check was based on thisAuthority using the port that the file system had. Now it with getCanonicalUri(), it uses default port. Is this correct? All checks seem to be using defaultPort.
#* Why remove?
{noformat}
if (thisAuthority == thatAuthority || // & authorities match
(thisAuthority != null &&
thisAuthority.equalsIgnoreCase(thatAuthority)))
return;
try { // or the default fs's uri
defaultUri = get(getConf()).getUri();
} catch (IOException e) {
throw new RuntimeException(e);
}
{noformat}
#* The new method does not work for some cases like the previous one. If thisAuthority and thatAuthority is null, the new mehod throw IllegalArgumentException. This is due to removing an if condition after scheme check.
#* If thisAuthority and thatAuthority are not null, you end up parsing the URI for thatAuthority twice. This is due to removing an if condition after scheme check.
# DistributedFileSystem#makeQualified remove makes me concerned. I have not gone through all the cases to make sure there is no problem with that change.
Can you please describe tests you run?
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2450) Only complete hostname is supported to
access data via hdfs://
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HDFS-2450:
------------------------------
Attachment: HDFS-2450.patch
I added more definite checks to {{FileSystem.checkPath(Path)}} to properly detect equivalence between long and short hostnames.
Tweak {{DistributedFileSystem}} to stop unnecessarily mangling the given URIs which led to the problem being noticed. This should prevent the DFS from not realizing it owns its paths, and not make the aforementioned {{checkPath}} changes have to work very hard.
Commit tests passed, will post full test results tomorrow morning.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450.patch
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Rajit Saha (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127936#comment-13127936 ]
Rajit Saha commented on HDFS-2450:
----------------------------------
This happens only with
hadoop.security.token.service.use_ip = false in core-site.xml in client side
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13131919#comment-13131919 ]
Hadoop QA commented on HDFS-2450:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12499910/HDFS-2450-1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1399//console
This message is automatically generated.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450.patch
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2450) Only complete hostname is supported to
access data via hdfs://
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HDFS-2450:
------------------------------
Attachment: HDFS-2450-3.patch
Moved assignment of temporaries per Suresh. Added an exhaustive set of tests.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450-3.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273387#comment-13273387 ]
Harsh J commented on HDFS-2450:
-------------------------------
Thank you for taking a look Daryn :)
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Fix For: 1.0.0
>
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450-3.patch, HDFS-2450-4.patch, HDFS-2450-5.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2450) Only complete hostname is supported to
access data via hdfs://
Posted by "Matt Foley (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Foley updated HDFS-2450:
-----------------------------
Fix Version/s: 0.20.205.1
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Fix For: 0.20.205.1
>
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450-3.patch, HDFS-2450-4.patch, HDFS-2450-5.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2450) Only complete hostname is supported to
access data via hdfs://
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HDFS-2450:
------------------------------
Attachment: HDFS-2450-4.patch
Reduced public visibility of {{FileSystem#getCanonicalUri()}}.
Shuffled the tests around.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450-3.patch, HDFS-2450-4.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Suresh Srinivas (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137677#comment-13137677 ]
Suresh Srinivas commented on HDFS-2450:
---------------------------------------
+1 for the patch.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450-3.patch, HDFS-2450-4.patch, HDFS-2450-5.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137690#comment-13137690 ]
Hadoop QA commented on HDFS-2450:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501178/HDFS-2450-5.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1478//console
This message is automatically generated.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450-3.patch, HDFS-2450-4.patch, HDFS-2450-5.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2450) Only complete hostname is supported to
access data via hdfs://
Posted by "Matt Foley (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Foley updated HDFS-2450:
-----------------------------
Target Version/s: 0.20.205.1
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450.patch
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2450) Only complete hostname is supported to
access data via hdfs://
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HDFS-2450:
------------------------------
Attachment: HDFS-2450-5.patch
Per suresh, renamed test resolver class, and upgraded {{FileSystem#getCanonicalUri()}} visibility to protected.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450-3.patch, HDFS-2450-4.patch, HDFS-2450-5.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported
to access data via hdfs://
Posted by "Suresh Srinivas (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136782#comment-13136782 ]
Suresh Srinivas commented on HDFS-2450:
---------------------------------------
Patch looks good. Some minor comments:
# Should the tests added in TestNetUtils for FileSystem#checkPath be moved to TestFileSystem?
# FileSytem#getCanonicalUri() is now a public method. Is this done for test purpose only or do you intend it to be a public API? If it is the former, then you could make it protected and addressing previous comment would allow calling it from TestFileSystem.
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
> Key: HDFS-2450
> URL: https://issues.apache.org/jira/browse/HDFS-2450
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Daryn Sharp
> Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450-3.patch, HDFS-2450.patch, IP vs. Hostname.pdf
>
>
> If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported
> $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira