You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Larry McCay (JIRA)" <ji...@apache.org> on 2017/01/13 19:17:26 UTC
[jira] [Updated] (HADOOP-13988) KMSClientProvider does not work
with WebHDFS and Apache Knox w/ProxyUser
[ https://issues.apache.org/jira/browse/HADOOP-13988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Larry McCay updated HADOOP-13988:
---------------------------------
Description:
After upgrading to HDP 2.5.3.0 noticed that all of the KMSClientProvider issues have not been resolved. We put a test build together and applied HADOOP-13558 and HADOOP-13749 these two fixes did still not solve the issue with requests coming from WebHDFS through to Knox to a TDE zone.
So we added some debug to our build and determined effectively what is happening here is a double proxy situation which does not seem to work. So we propose the following fix in getActualUgi Method:
{noformat}
}
// Use current user by default
UserGroupInformation actualUgi = currentUgi;
if (currentUgi.getRealUser() != null) {
// Use real user for proxy user
if (LOG.isDebugEnabled()) {
LOG.debug("using RealUser for proxyUser);
}
actualUgi = currentUgi.getRealUser();
if (getDoAsUser() != null) {
if (LOG.isDebugEnabled()) {
LOG.debug("doAsUser exists");
LOG.debug("currentUGI realUser shortName: {}", currentUgi.getRealUser().getShortUserName());
LOG.debug("processUGI loginUser shortName: {}", UserGroupInformation.getLoginUser().getShortUserName());
}
if (currentUgi.getRealUser().getShortUserName() != UserGroupInformation.getLoginUser().getShortUserName()) {
if (LOG.isDebugEnabled()) {
LOG.debug("currentUGI.realUser does not match UGI.processUser);
}
actualUgi = UserGroupInformation.getLoginUser();
if (LOG.isDebugEnabled()) {
LOG.debug("LoginUser for Proxy: {}", actualUgi.getLoginUser());
}
}
}
} else if (!currentUgiContainsKmsDt() &&
!currentUgi.hasKerberosCredentials()) {
// Use login user for user that does not have either
// Kerberos credential or KMS delegation token for KMS operations
if (LOG.isDebugEnabled()) {
LOG.debug("using loginUser no KMS Delegation Token no Kerberos Credentials");
}
actualUgi = currentUgi.getLoginUser();
}
return actualUgi;
}
{noformat}
was:
After upgrading to HDP 2.5.3.0 noticed that all of the KMSClientProvider issues have not been resolved. We put a test build together and applied HADOOP-13558 and HADOOP-13749 these two fixes did still not solve the issue with requests coming from WebHDFS through to Knox to a TDE zone.
So we added some debug to our build and determined effectively what is happening here is a double proxy situation which does not seem to work. So we propose the following fix in getActualUgi Method:
}
// Use current user by default
UserGroupInformation actualUgi = currentUgi;
if (currentUgi.getRealUser() != null) {
// Use real user for proxy user
if (LOG.isDebugEnabled()) {
LOG.debug("using RealUser for proxyUser);
}
actualUgi = currentUgi.getRealUser();
if (getDoAsUser() != null) {
if (LOG.isDebugEnabled()) {
LOG.debug("doAsUser exists");
LOG.debug("currentUGI realUser shortName: {}", currentUgi.getRealUser().getShortUserName());
LOG.debug("processUGI loginUser shortName: {}", UserGroupInformation.getLoginUser().getShortUserName());
}
if (currentUgi.getRealUser().getShortUserName() != UserGroupInformation.getLoginUser().getShortUserName()) {
if (LOG.isDebugEnabled()) {
LOG.debug("currentUGI.realUser does not match UGI.processUser);
}
actualUgi = UserGroupInformation.getLoginUser();
if (LOG.isDebugEnabled()) {
LOG.debug("LoginUser for Proxy: {}", actualUgi.getLoginUser());
}
}
}
} else if (!currentUgiContainsKmsDt() &&
!currentUgi.hasKerberosCredentials()) {
// Use login user for user that does not have either
// Kerberos credential or KMS delegation token for KMS operations
if (LOG.isDebugEnabled()) {
LOG.debug("using loginUser no KMS Delegation Token no Kerberos Credentials");
}
actualUgi = currentUgi.getLoginUser();
}
return actualUgi;
}
> KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser
> ------------------------------------------------------------------------
>
> Key: HADOOP-13988
> URL: https://issues.apache.org/jira/browse/HADOOP-13988
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, kms
> Affects Versions: 2.7.3
> Environment: HDP 2.5.3.0
> WebHDFSUser --> Knox --> HA NameNodes(WebHDFS) --> DataNodes
> Reporter: Greg Senia
>
> After upgrading to HDP 2.5.3.0 noticed that all of the KMSClientProvider issues have not been resolved. We put a test build together and applied HADOOP-13558 and HADOOP-13749 these two fixes did still not solve the issue with requests coming from WebHDFS through to Knox to a TDE zone.
> So we added some debug to our build and determined effectively what is happening here is a double proxy situation which does not seem to work. So we propose the following fix in getActualUgi Method:
> {noformat}
> }
> // Use current user by default
> UserGroupInformation actualUgi = currentUgi;
> if (currentUgi.getRealUser() != null) {
> // Use real user for proxy user
> if (LOG.isDebugEnabled()) {
> LOG.debug("using RealUser for proxyUser);
> }
> actualUgi = currentUgi.getRealUser();
> if (getDoAsUser() != null) {
> if (LOG.isDebugEnabled()) {
> LOG.debug("doAsUser exists");
> LOG.debug("currentUGI realUser shortName: {}", currentUgi.getRealUser().getShortUserName());
> LOG.debug("processUGI loginUser shortName: {}", UserGroupInformation.getLoginUser().getShortUserName());
> }
> if (currentUgi.getRealUser().getShortUserName() != UserGroupInformation.getLoginUser().getShortUserName()) {
> if (LOG.isDebugEnabled()) {
> LOG.debug("currentUGI.realUser does not match UGI.processUser);
> }
> actualUgi = UserGroupInformation.getLoginUser();
> if (LOG.isDebugEnabled()) {
> LOG.debug("LoginUser for Proxy: {}", actualUgi.getLoginUser());
> }
> }
> }
>
> } else if (!currentUgiContainsKmsDt() &&
> !currentUgi.hasKerberosCredentials()) {
> // Use login user for user that does not have either
> // Kerberos credential or KMS delegation token for KMS operations
> if (LOG.isDebugEnabled()) {
> LOG.debug("using loginUser no KMS Delegation Token no Kerberos Credentials");
> }
> actualUgi = currentUgi.getLoginUser();
> }
> return actualUgi;
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org