You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2019/06/10 16:42:00 UTC
[jira] [Comment Edited] (HADOOP-16354) Enable AuthFilter as default
for WebHdfs
[ https://issues.apache.org/jira/browse/HADOOP-16354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860140#comment-16860140 ]
Eric Yang edited comment on HADOOP-16354 at 6/10/19 4:41 PM:
-------------------------------------------------------------
[~Prabhu Joseph] Test case 2 is mixed for getting delegation token, and accessing via knox gateway. However, doAs flag is missing when requesting delegation token. Hence, the token returned from webhdfs is owned by Knox user instead of ambari-qa.
We can refine the test into two separate tests.
h2. 2.1 Knox obtain delegation token for end user for cross knox distcp
The test must be written as:
{code}
[knox@pjosephdocker-1 hadoop]$ curl --negotiate -u : "http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/?op=GETDELEGATIONTOKEN&renewer=hdfs&doAs=ambari-qa"
{"Token":{"urlString":"hash of delegation token for ambari-qa user"}}
{code}
{code}
[ambari-qa@pjosephdocker-1 ~]$ curl "http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/user/ambari-qa?op=GETFILESTATUS&delegation=hash of delegation token for ambari-qa user"
{code}
The key difference is in obtaining GETDELEGATIONTOKEN operation and doAs flag needs to work together for knox to obtain a valid toke for the end user. In the past, we allow doas= and also doAs=, this was a case insensitive flag.
h2. 2.2 Normal operation to get delegation token as end user for distcp
{code}
[ambari-qa@pjosephdocker-1 ~]$ curl --negotiate -u : "http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/?op=GETDELEGATIONTOKEN&renewer=hdfs"
{"Token":{"urlString":"IAAEa25veARoZGZzAIoBazYZx6CKAWtaJkugjgG_jgGkFDQ2gUTATHjMfowub5bl-SqLAwxmEldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjYuNzMuMTkwOjgwMjA"}}
{code}
{code}
[ambari-qa@pjosephdocker-1 ~]$ curl "http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/user/knox?op=GETFILESTATUS&delegation=IAAEa25veARoZGZzAIoBazYZx6CKAWtaJkugjgG_jgGkFDQ2gUTATHjMfowub5bl-SqLAwxmEldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjYuNzMuMTkwOjgwMjA"
{"FileStatus":{"accessTime":0,"blockSize":0,"childrenNum":0,"fileId":1394411,"group":"hadoop","length":0,"modificationTime":1559980208213,"owner":"knox","pathSuffix":"","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}
{code}
The test case 2.1 must work in AuthFilter regardless if ProxyUserAuthenticationFilter or AuthenticationFilter is configured to maintain backward compatibility.
was (Author: eyang):
[~Prabhu Joseph] Test case 2 is mixed for getting delegation token, and accessing via knox gateway. However, doAs flag is missing when requesting delegation token. Hence, the token returned from webhdfs is owned by Knox user instead of ambari-qa.
We can refine the test into two separate tests.
h2. 2.1 Knox obtain delegation token for end user for cross knox distcp
The test must be written as:
{code}
[knox@pjosephdocker-1 hadoop]$ curl --negotiate -u : "http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/?op=GETDELEGATIONTOKEN&renewer=hdfs&doAs=ambari-qa"
{"Token":{"urlString":"hash of delegation token for ambari-qa user"}}
{code}
{code}
[ambari-qa@pjosephdocker-1 ~]$ curl "http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/user/ambari-qa?op=GETFILESTATUS&delegation=hash of delegation token for ambari-qa user"
{code}
The key difference is in obtaining GETDELEGATIONTOKEN operation and doAs flag needs to work together for knox to obtain a valid toke for the end user. In the past, we allow doas= and also doAs=, this was a case insensitive flag.
h2. 2.2 Normal operation to get delegation token as end user for distcp
{code}
[ambari-qa@pjosephdocker-1 ~]$ curl --negotiate -u : "http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/?op=GETDELEGATIONTOKEN&renewer=hdfs"
{"Token":{"urlString":"IAAEa25veARoZGZzAIoBazYZx6CKAWtaJkugjgG_jgGkFDQ2gUTATHjMfowub5bl-SqLAwxmEldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjYuNzMuMTkwOjgwMjA"}}
{code}
{code}
[ambari-qa@pjosephdocker-1 ~]$ curl "http://pjosephdocker-1.openstacklocal:50070/webhdfs/v1/user/knox?op=GETFILESTATUS&delegation=IAAEa25veARoZGZzAIoBazYZx6CKAWtaJkugjgG_jgGkFDQ2gUTATHjMfowub5bl-SqLAwxmEldFQkhERlMgZGVsZWdhdGlvbhIxNzIuMjYuNzMuMTkwOjgwMjA"
{"FileStatus":{"accessTime":0,"blockSize":0,"childrenNum":0,"fileId":1394411,"group":"hadoop","length":0,"modificationTime":1559980208213,"owner":"knox","pathSuffix":"","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}
{code}
The test case 2.1 must work for in AuthFilter regardless if ProxyUserAuthenticationFilter or AuthenticationFilter is configured to maintain backward compatibility.
> Enable AuthFilter as default for WebHdfs
> ----------------------------------------
>
> Key: HADOOP-16354
> URL: https://issues.apache.org/jira/browse/HADOOP-16354
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: security
> Affects Versions: 3.3.0
> Reporter: Prabhu Joseph
> Assignee: Prabhu Joseph
> Priority: Major
> Attachments: HADOOP-16354-001.patch, HADOOP-16354-002.patch, HADOOP-16354-003.patch
>
>
> HADOOP-16314 provides an generic option to configure ProxyUserAuthenticationFilterInitializer (Kerberos + doAs support) for all the services. If this is not configured, AuthenticationFIlter is used for NameNode UI and WebHdfs. Will enable AuthFilter as default for WebHdfs so that it is backward compatible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org