You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Eli Collins (JIRA)" <ji...@apache.org> on 2012/09/27 06:01:10 UTC

[jira] [Created] (HDFS-3983) Hftp should not use the https port

Eli Collins created HDFS-3983:
---------------------------------

             Summary: Hftp should not use the https port
                 Key: HDFS-3983
                 URL: https://issues.apache.org/jira/browse/HDFS-3983
             Project: Hadoop HDFS
          Issue Type: Bug
    Affects Versions: 2.0.0-alpha
            Reporter: Eli Collins


Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:

{noformat}
$ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
ls: Security enabled but user not authenticated by filter
{noformat}

This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3983) Hftp should not use the https port

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465076#comment-13465076 ] 

Todd Lipcon commented on HDFS-3983:
-----------------------------------

Is it not already broken in trunk? In HDFS-2617, we removed all of the Krb5CipherSuites stuff, which would indicate that it won't work regardless.
                
> Hftp should not use the https port
> ----------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt, hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3983) Hftp should not use the https port

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464417#comment-13464417 ] 

Todd Lipcon commented on HDFS-3983:
-----------------------------------

+1 pending Jenkins, seems right to me given we're using SPNEGO auth instead of KSSL. Did you test hsftp as well?
                
> Hftp should not use the https port
> ----------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3983) Hftp should support both SPNEGO and KSSL

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465083#comment-13465083 ] 

Eli Collins commented on HDFS-3983:
-----------------------------------

Yes, it's already broken in trunk. HDFS-3699 tracks fixing this, and once that's in we'll still need to use the https port right? I'm removing the use of the https port entirely here, which we won't want to do. How about a smaller fix in the meantime that just defaults to the http port and we can make it use the https port per HDFS-3461 once HDFS-3699 is in?
                
> Hftp should support both SPNEGO and KSSL
> ----------------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt, hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3983) Hftp should not use the https port

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins updated HDFS-3983:
------------------------------

    Attachment: hdfs-3983.txt

Updated patch attached. Makes the same updates to TestHftpDelegationToken that TestHftpFileSystem required to reflect the new expected behavior. We use the configured http port (not the https port) and we no longer need to worry about using the secure port even if the user accesses hftp using an authority with port specified. The new behavior matches WebHDFS, we respect the port specified in the URI and use the http port by default.
                
> Hftp should not use the https port
> ----------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt, hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HDFS-3983) Hftp should not use the https port

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins reassigned HDFS-3983:
---------------------------------

    Assignee: Eli Collins
    
> Hftp should not use the https port
> ----------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3983) Hftp should not use the https port

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464521#comment-13464521 ] 

Hadoop QA commented on HDFS-3983:
---------------------------------

{color:green}+1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12546815/hdfs-3983.txt
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 2 new or modified test files.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of javac compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:green}+1 eclipse:eclipse{color}.  The patch built with eclipse:eclipse.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number of release audit warnings.

    {color:green}+1 core tests{color}.  The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

    {color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3241//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3241//console

This message is automatically generated.
                
> Hftp should not use the https port
> ----------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt, hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3983) Hftp should support both SPNEGO and KSSL

Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465085#comment-13465085 ] 

Alejandro Abdelnur commented on HDFS-3983:
------------------------------------------

IMO we should use a different scheme (via a new FileSystem client), i.e. *webhdfss* for the following reasons:

* The URLs themselves will indicate if the resource is served over a secure transport or not.
* A client may need to communicate with both webhdfs:// and webhdfss:// endpoints, thus a client config will not cut it.

                
> Hftp should support both SPNEGO and KSSL
> ----------------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt, hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3983) Hftp should not use the https port

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465073#comment-13465073 ] 

Eli Collins commented on HDFS-3983:
-----------------------------------

Btw I'm going to hold off on committing this patch for now because like HDFS-3461 it needs to support KSSL for getting the delegation token. We need this to support distcp from v1 using hftp and security.
                
> Hftp should not use the https port
> ----------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt, hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3983) Hftp should not use the https port

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464476#comment-13464476 ] 

Hadoop QA commented on HDFS-3983:
---------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12546800/hdfs-3983.txt
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 1 new or modified test files.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of javac compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:green}+1 eclipse:eclipse{color}.  The patch built with eclipse:eclipse.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number of release audit warnings.

    {color:red}-1 core tests{color}.  The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

                  org.apache.hadoop.hdfs.TestHftpDelegationToken

    {color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3240//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3240//console

This message is automatically generated.
                
> Hftp should not use the https port
> ----------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt, hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3983) Hftp should not use the https port

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins updated HDFS-3983:
------------------------------

    Status: Patch Available  (was: Open)
    
> Hftp should not use the https port
> ----------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3983) Hftp should not use the https port

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465007#comment-13465007 ] 

Eli Collins commented on HDFS-3983:
-----------------------------------

By having Hftp and WebHdfs support https (via HADOOP-8581). Ie have one FileSystem implementation support both schemes.
                
> Hftp should not use the https port
> ----------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt, hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3983) Hftp should support both SPNEGO and KSSL

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins updated HDFS-3983:
------------------------------

    Component/s: security
        Summary: Hftp should support both SPNEGO and KSSL  (was: Hftp should not use the https port)
    
> Hftp should support both SPNEGO and KSSL
> ----------------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt, hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3983) Hftp should not use the https port

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465064#comment-13465064 ] 

Eli Collins commented on HDFS-3983:
-----------------------------------

Actually Tucu pointed out offline that you need two classes because clients like curl (for webhdfs) can't check the hadoop.ssl.enabled flag. So basically there are two things here:
# Add a "webhdfss" that supports SSL
# Make sure Hsftp works with the SSL server provided by HADOOP-8581 and remove the duplicate SSL listener code in the NN (HttpServer and NameNodeHttpServer) 
                
> Hftp should not use the https port
> ----------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt, hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3983) Hftp should not use the https port

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464984#comment-13464984 ] 

Todd Lipcon commented on HDFS-3983:
-----------------------------------

If we remove hsftp, how do we do a cross-datacenter copy where the link between them is untrusted?
                
> Hftp should not use the https port
> ----------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt, hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3983) Hftp should not use the https port

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464904#comment-13464904 ] 

Eli Collins commented on HDFS-3983:
-----------------------------------

Btw per your comment wrt Hsftp, I haven't tested it. I don't think it works or has for a while, confirmed offline a couple other people thought the same am going to start a thread about removing it.
                
> Hftp should not use the https port
> ----------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt, hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3983) Hftp should not use the https port

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins updated HDFS-3983:
------------------------------

    Status: Open  (was: Patch Available)
    
> Hftp should not use the https port
> ----------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt, hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3983) Hftp should not use the https port

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins updated HDFS-3983:
------------------------------

    Attachment: hdfs-3983.txt

Patch attached. I verified on a secure cluster that hftp now works w/o setting dfs.https.port to the http port.
                
> Hftp should not use the https port
> ----------------------------------
>
>                 Key: HDFS-3983
>                 URL: https://issues.apache.org/jira/browse/HDFS-3983
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3983.txt
>
>
> Hftp currently doesn't work against a secure cluster unless you configure {{dfs.https.port}} to be the http port, otherwise the client can't fetch tokens:
> {noformat}
> $ hadoop fs -ls hftp://c1225.hal.cloudera.com:50070/
> 12/09/26 18:02:00 INFO fs.FileSystem: Couldn't get a delegation token from http://c1225.hal.cloudera.com:50470 using http.
> ls: Security enabled but user not authenticated by filter
> {noformat}
> This is due to Hftp still using the https port. Post HDFS-2617 it should use the regular http port. Hsftp should still use the secure port, however now that we have HADOOP-8581 it's worth considering removing Hsftp entirely. I'll start a separate thread about that.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira