[jira] [Commented] (KNOX-482) Support DistCp via Knox

["Kevin Minder (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KNOX-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260176#comment-14260176 ] 

Kevin Minder commented on KNOX-482:
-----------------------------------

Attached is the patch containing the changes (ie hacks) I made to get DistCp to work in secure mode via Knox.  So many things that need real solutions here...  Also attached the relevant config files for reference. 

The one especially weird thing is this in core-site.xml which was required to allow Knox to participate in a delegation token exchanges between the YARN ResourceManager and HDFS.  I'm not sure how acceptable this will be in the field.

    <property>
      <name>hadoop.proxyuser.knox.groups</name>
      <value>users,hadoop</value>
    </property>


The change in HadoopAuthPostFilter is probably a valid one in that it insures the full Kerberos principal (e.g. ambari-qa/c6401.ambari.apache.org@EXAMPLE.COM) is used when present.

There are two really bad hacks in the patch.

    In WebHdfsDepoymentContributor, the filter chain for the DN URLs has all but the rewrite and dispatch providers removed.  This assumes that the DN will protect itself by requiring a block access tokens.  Of special concern would be what this means for DN UI URLs.
    For access to NN if there is a delegation token the token itself is used as the principal.  This will certainly cause issues for down stream processing that assumes that this will be a real user principal.
        In IdentityAsserterHttpServletRequestWrapper if there is a delegation token that is used as the actual principal name. 
        In HadoopAuthFilter, if there is a delegation token present a Subject is created with the value of the token used as the PrimparyPrincipal. 

> Support DistCp via Knox
> -----------------------
>
>                 Key: KNOX-482
>                 URL: https://issues.apache.org/jira/browse/KNOX-482
>             Project: Apache Knox
>          Issue Type: New Feature
>          Components: Server
>    Affects Versions: 0.6.0
>            Reporter: Kevin Minder
>             Fix For: 0.6.0
>
>         Attachments: core-site.xml, default.xml, distcp-poc.patch, gateway-site.xml
>
>
> Support the use of Knox in hadoop distcp use cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)