You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "zhiyong zhang (JIRA)" <ji...@apache.org> on 2009/05/15 20:53:45 UTC
[jira] Created: (HADOOP-5851) proxy to call LDAP for IP lookup and
get user ID and directories, validate requested URL
proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
----------------------------------------------------------------------------------------
Key: HADOOP-5851
URL: https://issues.apache.org/jira/browse/HADOOP-5851
Project: Hadoop Core
Issue Type: New Feature
Components: contrib/hdfsproxy
Reporter: zhiyong zhang
Assignee: zhiyong zhang
Priority: Critical
It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5851) proxy to call LDAP for IP lookup and
get user ID and directories, validate requested URL
Posted by "zhiyong zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhiyong zhang updated HADOOP-5851:
----------------------------------
Attachment: HADOOP-5851.patch
a) add another filter code named LdapIpDirFilter.java, will be installed as a deployment descriptor (in web.xml).
b) program flowchart.
1. IP = HttpServletRequest.getRemoteAddr()
2. Ldap.search(uniqueMember:cn=IP) to find role.
2.a. if nothing found, 403 return
3. role A was found, from role A, extract userId and HDFS root paths allowed, say PLIST.
4. log userId.
5. get PATH = HttpServletRequest.getPathInfo()
6. check if PATH or its parents is in PLIST
6.a. if not, 403 return.
7. access allowed, filter pass
c) use a Dummy LDAP Object server to do the unit test.
> proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-5851
> URL: https://issues.apache.org/jira/browse/HADOOP-5851
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: zhiyong zhang
> Assignee: zhiyong zhang
> Priority: Critical
> Attachments: HADOOP-5851.patch
>
>
> It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
> when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5851) proxy to call LDAP for IP lookup and
get user ID and directories, validate requested URL
Posted by "zhiyong zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhiyong zhang updated HADOOP-5851:
----------------------------------
Attachment: HADOOP-5851.patch
corrected several typos and changed several places for code optimization.
> proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-5851
> URL: https://issues.apache.org/jira/browse/HADOOP-5851
> Project: Hadoop Common
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: zhiyong zhang
> Assignee: zhiyong zhang
> Priority: Critical
> Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch
>
>
> It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
> when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5851) proxy to call LDAP for IP lookup
and get user ID and directories, validate requested URL
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716397#action_12716397 ]
Kan Zhang commented on HADOOP-5851:
-----------------------------------
@zhiyong, why you want to do that? If SSL authentication is not needed, one can use HFTP instead of HSFTP.
> proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-5851
> URL: https://issues.apache.org/jira/browse/HADOOP-5851
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: zhiyong zhang
> Assignee: zhiyong zhang
> Priority: Critical
> Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch
>
>
> It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
> when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5851) proxy to call LDAP for IP lookup and
get user ID and directories, validate requested URL
Posted by "zhiyong zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhiyong zhang updated HADOOP-5851:
----------------------------------
Attachment: HADOOP-5851.patch
remove find-bug warnings.
> proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-5851
> URL: https://issues.apache.org/jira/browse/HADOOP-5851
> Project: Hadoop Common
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: zhiyong zhang
> Assignee: zhiyong zhang
> Priority: Critical
> Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch
>
>
> It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
> when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5851) proxy to call LDAP for IP lookup and
get user ID and directories, validate requested URL
Posted by "zhiyong zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhiyong zhang updated HADOOP-5851:
----------------------------------
Attachment: HADOOP-5851.patch
use hostname.split("[-\\.]") to extract servlet context path (war file name)
> proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-5851
> URL: https://issues.apache.org/jira/browse/HADOOP-5851
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: zhiyong zhang
> Assignee: zhiyong zhang
> Priority: Critical
> Attachments: HADOOP-5851.patch, HADOOP-5851.patch
>
>
> It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
> when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5851) proxy to call LDAP for IP lookup
and get user ID and directories, validate requested URL
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716401#action_12716401 ]
Kan Zhang commented on HADOOP-5851:
-----------------------------------
I can see that for encryption purpose you may still want to use SSL. In that case, I suggest you make it an option instead of the default behavior. The user has to explicitly specify this option to enable it.
> proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-5851
> URL: https://issues.apache.org/jira/browse/HADOOP-5851
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: zhiyong zhang
> Assignee: zhiyong zhang
> Priority: Critical
> Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch
>
>
> It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
> when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5851) proxy to call LDAP for IP lookup
and get user ID and directories, validate requested URL
Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12723363#action_12723363 ]
Philip Zeyliger commented on HADOOP-5851:
-----------------------------------------
I'm traveling overseas and will return on June 29th. For urgent
matters, please contact Amr Awadallah.
> proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-5851
> URL: https://issues.apache.org/jira/browse/HADOOP-5851
> Project: Hadoop Common
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: zhiyong zhang
> Assignee: zhiyong zhang
> Priority: Critical
> Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch
>
>
> It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
> when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5851) proxy to call LDAP for IP lookup and
get user ID and directories, validate requested URL
Posted by "zhiyong zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhiyong zhang updated HADOOP-5851:
----------------------------------
Attachment: HADOOP-5851.patch
in client side (HsftpFileSystem.java), use a DummyTrustStoreManager to accept all server certificates, in case
ssl-client.xml is not defined.
like curl's -k option.
use this client, user will not need to worry about importing server CA to truststore.
> proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-5851
> URL: https://issues.apache.org/jira/browse/HADOOP-5851
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: zhiyong zhang
> Assignee: zhiyong zhang
> Priority: Critical
> Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch
>
>
> It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
> when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5851) proxy to call LDAP for IP lookup and
get user ID and directories, validate requested URL
Posted by "zhiyong zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhiyong zhang updated HADOOP-5851:
----------------------------------
Attachment: HADOOP-5851.patch
1.) As suggested by Kan, trust all server certs should not be the default setting.
Add
<property>
<name>ssl.client.do.not.authenticate.server</name>
<value>false</value>
<description> if true, trust all server certificates
</description>
</property>
in ssl-client.xml configuration. By default, still need to validate server certificates.
2.) Did a code walk-through with Rob. W. Also discussed with Kan, the should get the group information from LDAP server instead of using proxyUgiManager. changed this part so that proxy will pass HDFS userId and group info to source cluster.
3.) Merged all configuration files into one configuration file (hdfsproxy-default.xml), this saves some effort to manage the configuration files and war files.
The configuration in hdfsproxy-default.xml should contain
<name>fs.default.name</name>
<name>dfs.block.size</name>
<name>io.file.buffer.size</name>
in addition to ldap-based properties.
> proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-5851
> URL: https://issues.apache.org/jira/browse/HADOOP-5851
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: zhiyong zhang
> Assignee: zhiyong zhang
> Priority: Critical
> Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch
>
>
> It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
> when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5851) proxy to call LDAP for IP lookup
and get user ID and directories, validate requested URL
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12723360#action_12723360 ]
Chris Douglas commented on HADOOP-5851:
---------------------------------------
The patch needs to be regenerated for the code split.
The contents look fine, though I'd like to see the TrustManager path completely replace the System.setProperty idiom now that it is understood. The only other minor nit would be here:
{noformat}
+ if (LOG.isDebugEnabled())
+ e.printStackTrace();
{noformat}
Using {{LOG.debug("Useful message", e)}} is more standard.
> proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-5851
> URL: https://issues.apache.org/jira/browse/HADOOP-5851
> Project: Hadoop Common
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: zhiyong zhang
> Assignee: zhiyong zhang
> Priority: Critical
> Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch
>
>
> It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
> when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5851) proxy to call LDAP for IP lookup and
get user ID and directories, validate requested URL
Posted by "zhiyong zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhiyong zhang updated HADOOP-5851:
----------------------------------
Attachment: HADOOP-5851.patch
1. removed System.setProperty. Use Keymanagers and TrustManagers instead.
2. re-patched in new hdfs trunk after project split.
3. replaced the printStackTrace with normal LOG.debug(e.toString()).
> proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-5851
> URL: https://issues.apache.org/jira/browse/HADOOP-5851
> Project: Hadoop Common
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: zhiyong zhang
> Assignee: zhiyong zhang
> Priority: Critical
> Attachments: HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch, HADOOP-5851.patch
>
>
> It is easy to manage user accounts using LDAP. by adding support for LDAP, proxy can do IP authorization in a headless fashion.
> when a user send a request, proxy extract IP address and request PathInfo from the request. then it searches the LDAP server to get the allowed HDFS root paths given the IP address. Proxy will match the user request PathInfo with the allowed HDFS root path, return 403 if it could not find a match.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.