You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Ivan Mitic (JIRA)" <ji...@apache.org> on 2012/05/19 00:43:06 UTC
[jira] [Created] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Ivan Mitic created HADOOP-8414:
----------------------------------
Summary: Address problems related to localhost resolving to 127.0.0.1 on Windows
Key: HADOOP-8414
URL: https://issues.apache.org/jira/browse/HADOOP-8414
Project: Hadoop Common
Issue Type: Bug
Components: fs, test
Affects Versions: 1.0.0
Reporter: Ivan Mitic
Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
- TestHarFileSystem
- TestCLI
- TestSaslRPC
This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280573#comment-13280573 ]
Daryn Sharp commented on HADOOP-8414:
-------------------------------------
Odd, for my known knowledge of quirks, does {{InetAddress.getByName(null).getHostName()}} and {{InetAddress.getByName(null).getCanonicalHostName()}} also return just 127.0.0.1? This is a bit dismaying, because the tests are trying to verify that the output contains the _exact path(s)_ as given on the cmdline. I spent quite a bit of time ensuring one can parse the output for a given path. Allowing either localhost or 127.0.0.1 to match means the tests won't catch if something mangles the host for non-windows platforms. I'm not sure how to deal with that... :(
The majority of the code mangling the authority into another uri, that you had to touch, is basically trying to convert {{scheme-host:port}} into {{scheme://host:port}}. Instead of all the null and -1 checks to rebuild the uri, it seems it would be simpler to just use {{replaceFirst}}. It also avoids introducing more ipv6 incompatibilities caused by a naive split on ":".
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401056#comment-13401056 ]
Ivan Mitic commented on HADOOP-8414:
------------------------------------
bq. The patch still allows a different authority to be returned than the one actually provided. If implemented correctly, the test pattern matches won't require any changes. I'll try to take a swag at a small patch that will make unix/windows behavior equivalent.
Daryn, did you maybe get a chance to look into this? Would it be fine to proceed with this patch as is, and file a separate Jira to track the investigation on possible improvements? Thanks!
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win(2).patch, HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396881#comment-13396881 ]
Daryn Sharp commented on HADOOP-8414:
-------------------------------------
The patch still allows a different authority to be returned than the one actually provided. If implemented correctly, the test pattern matches won't require any changes. I'll try to take a swag at a small patch that will make unix/windows behavior equivalent.
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win(2).patch, HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281593#comment-13281593 ]
Daryn Sharp commented on HADOOP-8414:
-------------------------------------
bq. For InetAddress.getByName(), the two actually return different values, "localhost" and "127.0.0.1", respectively.
Good, that is exactly what I would expect! However this makes it even more odd that {{new InetSocketAddress("localhost", port).getHostName()}} on windows will return {{127.0.0.1}} instead of {{localhost}}... The {{InetSocketAddress}} ctor is supposed to call {{InetAddress.getByName(host)}} and then {{InetSocketAddress#getHostName()}} delegates to {{InetAddress#getHostName()}}. I would expect {{InetAddress.getByName(hostname)}} to behave similarly for {{localhost}} and {{null}} as the given hostname.
Could you please do the same testing with {{InetAddress.getByName("localhost")}} to ensure that something else isn't botching the hostname to {{127.0.0.1}}?
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8414:
-------------------------------
Attachment: HADOOP-8414-branch-1-win.patch
Attaching patch.
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281752#comment-13281752 ]
Ivan Mitic commented on HADOOP-8414:
------------------------------------
Thanks Daryn, some answers/comments inline.
Could you please do the same testing with InetAddress.getByName("localhost") to ensure that something else isn't botching the hostname to 127.0.0.1?
@Daryn: Same result as with null.
The HarFileSystem change is probably worthy of its own jira now since it's generally useful and should be able to stand alone w/o the other changes in this jira
@Daryn: Sure.
I forgot to mention that if the discrepancy with window's resolution of localhost is likely to break host-based tokens (hadoop.security.token.service.use_ip=false). We'll need to be sure to test host-based tokens if the discrepancy is real and can't be fixed.
@Daryn: I see, thanks for bringing this up. Do you maybe know of a test that can catch this?
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281600#comment-13281600 ]
Daryn Sharp commented on HADOOP-8414:
-------------------------------------
The {{HarFileSystem}} change is probably worthy of its own jira now since it's generally useful and should be able to stand alone w/o the other changes in this jira. This jira would then just depend on the new one since it indirectly addresses issues for this jira.
I forgot to mention that if the discrepancy with window's resolution of {{localhost}} is likely to break host-based tokens ({{hadoop.security.token.service.use_ip=false}}). We'll need to be sure to test host-based tokens if the discrepancy is real and can't be fixed.
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8414:
-------------------------------
Attachment: HADOOP-8414-branch-1-win(2).patch
Attaching updated patch.
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win(2).patch, HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406016#comment-13406016 ]
Daryn Sharp commented on HADOOP-8414:
-------------------------------------
+1
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win(2).patch, HADOOP-8414-branch-1-win(3).patch, HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402428#comment-13402428 ]
Ivan Mitic commented on HADOOP-8414:
------------------------------------
Thanks Daryn, looking forward to your response!
In the meantime, I was thinking about one possible way to address this, and you let me know if it makes sense. Currently, the test assumes that rDNS will resolve {{127.0.0.1}} to {{localhost}}, what turned out not to be always true. We can change the test result to have a {{NAMENODE}} embedded in it, and then, in test code, replace {{NAMENODE}} with a constant value that was used as input. Now, the input being {{localhost}} or {{127.0.0.1}}, we are verifying that the output is equivalent to input and only input.
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win(2).patch, HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Bikas Saha (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bikas Saha updated HADOOP-8414:
-------------------------------
Assignee: Ivan Mitic (was: Chuan Liu)
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403145#comment-13403145 ]
Daryn Sharp commented on HADOOP-8414:
-------------------------------------
Maybe the details of this jira got paged out to faulty media... My understanding is the test is passing in "localhost" uris, but windows mangles them to contain the IP address which causes output != input? Reverse dns isn't supposed to be occurring at all, so windows appears to not "remember" the given hostname used to instantiate the {{InetAddress}}/{{InetSocketAddress}}. If true, I don't see how using the NAMENODE var would help?
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win(2).patch, HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402277#comment-13402277 ]
Daryn Sharp commented on HADOOP-8414:
-------------------------------------
I'm very sorry, I planned to work on this last week but a oozie/proxy-token/hftp blocker came up... I'll get to it by EOW.
After all the time I spent getting the shell path display correct, I'm reluctant to introduce inconsistent behavior on windows whose test changes may allow a breakage in non-windows to go undetected.
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win(2).patch, HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393394#comment-13393394 ]
Ivan Mitic commented on HADOOP-8414:
------------------------------------
bq. The FsShell or FileSystem shouldn't be using the canonical host name or even mangling the URI at all unless a change recently slipped past me...
@Daryn: Thanks for clarifying your concerns. Commands passed in by the test have the following form: {code}-fs NAMENODE -du /data15bytes{code} where the {{NAMENODE}} string gets replaced with the {{fs.default.name}}. Per my previous comment, namenode default FS URI resolves to {{hdfs://127.0.0.1:port}} on Windows, so from the FsShell perspective, output is equivalent to input.
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win(2).patch, HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280498#comment-13280498 ]
Daryn Sharp commented on HADOOP-8414:
-------------------------------------
Also, couldn't a lot of that logic be obliterated by doing something like:
{code}URI.create(rawUri.getAuthority().replaceFirst("-", "://")){code}
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8414:
-------------------------------
Attachment: HADOOP-8414-branch-1-win(3).patch
Thanks for commenting Daryn, it seems that there was a disconnect then :)
On Windows, {{TestCLI#namenode}} string resolves to {{hdfs://127.0.0.1:XXXX}}, and from this point on, all inputs to FsShell that have {{NAMENODE}} in it, will be expanded with {{hdfs://127.0.0.1:XXXX}}, causing the comparison to fail because the expected output is something with {{hdfs://localhost:XXXX}}. The output is actually equivalent to input on Windows.
Attaching the new patch . Now the expected output will always be in line with what is being sent as an input. Hope it looks better now!
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win(2).patch, HADOOP-8414-branch-1-win(3).patch, HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286989#comment-13286989 ]
Ivan Mitic commented on HADOOP-8414:
------------------------------------
Thanks, I'll take a look at this.
Btw, I separated the Har file system change into a separate patch [HADOOP-8440|https://issues.apache.org/jira/browse/HADOOP-8440] in case you want to take a look.
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Sanjay Radia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sanjay Radia updated HADOOP-8414:
---------------------------------
Assignee: Ivan Mitic
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406009#comment-13406009 ]
Ivan Mitic commented on HADOOP-8414:
------------------------------------
Great to hear that the latest patch looks good.
bq. Are you sure there isn't an existing expansion of NAMENODE in the output? I thought I added that functionality last year...?
I could not find this in branch-1. Just to make sure, I commented out the expansion I added and the test fails. Let me know if you have any additional comments.
Thanks!
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win(2).patch, HADOOP-8414-branch-1-win(3).patch, HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Matt Foley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Foley updated HADOOP-8414:
-------------------------------
Target Version/s: 1-win (was: 1.1.0)
Fix Version/s: 1-win
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Fix For: 1-win
>
> Attachments: HADOOP-8414-branch-1-win(2).patch, HADOOP-8414-branch-1-win(3).patch, HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Sanjay Radia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sanjay Radia resolved HADOOP-8414.
----------------------------------
Resolution: Fixed
Thanks Ivan, committed to branch-1 windows.
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win(2).patch, HADOOP-8414-branch-1-win(3).patch, HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286558#comment-13286558 ]
Daryn Sharp commented on HADOOP-8414:
-------------------------------------
The best I can think of is to modify {{NetUtils}} to do a {{addStaticResolution}} with the result of {{getByName(null)}}.
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281860#comment-13281860 ]
Ivan Mitic commented on HADOOP-8414:
------------------------------------
bq. If however #2 is also returning localhost as expected, then it means there's probably something wrong in FsShell or DistributedFileSystem's handling of the URI.
Oh, sorry for not being clear, below is the output:
InetAddress.getByName(null).getHostName() == localhost
InetAddress.getByName(null).getCanonicalHostName() == 127.0.0.1
InetAddress.getByName("localhost").getHostName() == localhost
InetAddress.getByName("localhost").getCanonicalHostName() == 127.0.0.1
new InetSocketAddress("localhost", 8000).getHostName() == localhost
PS. thanks for the tip.
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280489#comment-13280489 ]
Daryn Sharp commented on HADOOP-8414:
-------------------------------------
I'm not sure I understand the issue. Are you saying that on Windows creating an {{InetSocketAddress}} from "localhost" will return 127.0.0.1 as the hostname?
Also, isn't it a bit redundant to check both host and authority? Why not just authority?
{code}if (uri.getHost() == null && uri.getAuthority() == null){code}
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Bikas Saha (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bikas Saha updated HADOOP-8414:
-------------------------------
Assignee: Chuan Liu (was: Ivan Mitic)
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Chuan Liu
> Attachments: HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13405981#comment-13405981 ]
Daryn Sharp commented on HADOOP-8414:
-------------------------------------
Ah, yes, there was a disconnect. I think the latest patch looks good. Are you sure there isn't an existing expansion of NAMENODE in the output? I thought I added that functionality last year...?
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win(2).patch, HADOOP-8414-branch-1-win(3).patch, HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8414:
-------------------------------
Attachment: HADOOP-8414-branch-1-win.patch
Attaching updated patch.
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13283733#comment-13283733 ]
Ivan Mitic commented on HADOOP-8414:
------------------------------------
Thanks for the pointers Daryn! After looking more into this, I came to the following:
InetAddress.getByName("127.0.0.1").getHostName() == localhost (on Linux)
InetAddress.getByName("127.0.0.1").getHostName() == 127.0.0.1 (on Windows)
Going further, namenode's default filesystem URI is created using the above host name, and this URI is later used for DFS path resolution.
>From looking at the documentation for InetAddress.getHostName(), it is supposed to perform a reverse DNS lookup of the address. For some reason, this does not work well for 127.0.0.1 in Java under Windows. I tried a few other IP addresses, and getByName(IP).getHostName() worked fine.
Do you maybe have other suggestions/alternatives we can use to address the problem?
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281884#comment-13281884 ]
Daryn Sharp commented on HADOOP-8414:
-------------------------------------
Hmm, that's odd that {{getCanonicalHostName}} is returning an IP. Ignoring that, the other cases all look perfect so there's a grue lurking somewhere. The {{FsShell}} or {{FileSystem}} shouldn't be using the canonical host name or even mangling the URI at all unless a change recently slipped past me...
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280527#comment-13280527 ]
Ivan Mitic commented on HADOOP-8414:
------------------------------------
Thanks for the feedback Daryn. Answers below.
Daryn:I'm not sure I understand the issue. Are you saying that on Windows creating an InetSocketAddress from "localhost" will return 127.0.0.1 as the hostname?
Ivan: Correct.
Daryn:Also, isn't it a bit redundant to check both host and authority? Why not just authority?
Ivan: Agree, will fix this.
Not sure I understand your proposal around replaceFirst("-", "://")). Can you please elaborate?
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281270#comment-13281270 ]
Ivan Mitic commented on HADOOP-8414:
------------------------------------
Thanks Daryn!
For InetAddress.getByName(), the two actually return different values, "localhost" and "127.0.0.1", respectively.
Thanks for clarifying the replaceFirst("-", "://") part. I moved to this model and indeed it is simpler. Will submit an updated patch soon.
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281827#comment-13281827 ]
Daryn Sharp commented on HADOOP-8414:
-------------------------------------
{quote}bq. Could you please do the same testing with InetAddress.getByName("localhost") to ensure that something else isn't botching the hostname to 127.0.0.1?
@Daryn: Same result as with null.{quote}
If I understand correctly:
# {{InetAddress.getByName("localhost").getHostName()}} returns {{localhost}}
# {{new InetSocketAddress("localhost", port).getHostName()}} returns {{127.0.0.1}}??
If however #2 is also returning {{localhost}} as expected, then it means there's probably something wrong in {{FsShell}} or {{DistributedFileSystem}}'s handling of the URI.
As for host-based tokens, {{TestSecurityUtil}} _should_ catch problems, but there's a jira where I have to bring over a slew of more tests from 205 that would be more definitive.
PS. You can use "bq." or "\{quote\}" on prior statements. It's a bit easier to read than inline responses. If you click the (?), there's a lot of simple and useful formatting available.
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch, HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8414) Address problems related to
localhost resolving to 127.0.0.1 on Windows
Posted by "Bikas Saha (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280374#comment-13280374 ]
Bikas Saha commented on HADOOP-8414:
------------------------------------
+1 looks good
> Address problems related to localhost resolving to 127.0.0.1 on Windows
> -----------------------------------------------------------------------
>
> Key: HADOOP-8414
> URL: https://issues.apache.org/jira/browse/HADOOP-8414
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
> Affects Versions: 1.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8414-branch-1-win.patch
>
>
> Localhost resolves to 127.0.0.1 on Windows and that causes the following tests to fail:
> - TestHarFileSystem
> - TestCLI
> - TestSaslRPC
> This Jira tracks fixing these tests and other possible places that have similar issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira