You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@knox.apache.org by "handavid (via GitHub)" <gi...@apache.org> on 2026/03/13 17:32:31 UTC
[PR] KNOX-3277. LDAP Proxy improvements for working with AD backend [knox]
handavid opened a new pull request, #1177:
URL: https://github.com/apache/knox/pull/1177
The LdapProxyBackend and GroupLookupInterceptors are updated to work with sAMAccountName in addition to uid and cn for user and group lookup. This involved adding the memberOf and sAMAccountName attribute types to the schema used by the proxy.
Group retrieval during user search is fixed to respect the useMemberOf flag.
Tests for the LdapProxyBackend were added.
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
## How was this patch tested?
Changes were manually tested by running `ant start-test-gateway` configured against both AD and the test LDAP server.
Unit tests were added to cover the LdapProxyBackend behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KNOX-3277 : LDAP Proxy improvements for working with AD backend [knox]
Posted by "handavid (via GitHub)" <gi...@apache.org>.
handavid commented on code in PR #1177:
URL: https://github.com/apache/knox/pull/1177#discussion_r2954835031
##########
gateway-server/src/main/java/org/apache/knox/gateway/services/ldap/backend/LdapProxyBackend.java:
##########
@@ -63,6 +64,14 @@ public class LdapProxyBackend implements LdapBackend {
private String groupMemberAttribute = "memberUid"; // member for AD, memberUid for POSIX
private boolean useMemberOf; // Use memberOf attribute for group lookup (efficient for AD)
+ private List<String> proxyEntityAttributeTypes = List.of(
+ // "uid" will always be filled
+ "cn",
+ "dn",
Review Comment:
there are two schemas in play. The `ldap-proxy-backend-test.ldif` schema is for the backend ldap server (i.e., the real server). The `proxyEntityAttributeTypes` are those entities that will be copied and returned by the proxy ldap server. These two lists do not need to match exactly.
In this PR, the list is factored up from the `createProxyEntry` method for easier inspection and future modification. This is not a change from the existing behavior of the ldap proxy. We'll probably extend this list or pull it out to configuration in the future to allow users to define schema for proxy entities.
I just noticed the name mismatch between `Entity` and `Entry`. I'll fix the name of this list to `proxyEntryAttributeTypes`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KNOX-3277 : LDAP Proxy improvements for working with AD backend [knox]
Posted by "handavid (via GitHub)" <gi...@apache.org>.
handavid commented on code in PR #1177:
URL: https://github.com/apache/knox/pull/1177#discussion_r2955111151
##########
gateway-server/src/main/java/org/apache/knox/gateway/services/ldap/backend/LdapProxyBackend.java:
##########
@@ -445,7 +459,11 @@ private void copyAttribute(Entry source, Entry target, String attributeName) thr
if (attr != null) {
// Copy all values of the attribute (important for multi-valued attributes like objectClass)
for (org.apache.directory.api.ldap.model.entry.Value value : attr) {
- target.add(attributeName, value.getString());
+ try {
+ target.add(attributeName, value.getString());
+ } catch (Exception e) {
+ LOG.ldapAttributeCopyError(e);
Review Comment:
added rethrow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KNOX-3277 : LDAP Proxy improvements for working with AD backend [knox]
Posted by "handavid (via GitHub)" <gi...@apache.org>.
handavid commented on PR #1177:
URL: https://github.com/apache/knox/pull/1177#issuecomment-4084344352
@moresandeep I've updated the PR based on your comments. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KNOX-3277. LDAP Proxy improvements for working with AD backend [knox]
Posted by "handavid (via GitHub)" <gi...@apache.org>.
handavid commented on PR #1177:
URL: https://github.com/apache/knox/pull/1177#issuecomment-4056793899
@lmccay
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KNOX-3277 : LDAP Proxy improvements for working with AD backend [knox]
Posted by "handavid (via GitHub)" <gi...@apache.org>.
handavid commented on code in PR #1177:
URL: https://github.com/apache/knox/pull/1177#discussion_r2954853708
##########
gateway-server/src/main/java/org/apache/knox/gateway/services/ldap/backend/LdapProxyBackend.java:
##########
@@ -445,7 +459,11 @@ private void copyAttribute(Entry source, Entry target, String attributeName) thr
if (attr != null) {
// Copy all values of the attribute (important for multi-valued attributes like objectClass)
for (org.apache.directory.api.ldap.model.entry.Value value : attr) {
- target.add(attributeName, value.getString());
+ try {
+ target.add(attributeName, value.getString());
+ } catch (Exception e) {
+ LOG.ldapAttributeCopyError(e);
Review Comment:
right. I was conflicted about this behavior also. The problem I was originally solving was that the entry is omitted from search results if we request an attribute that isn't defined in the proxy schema. Let me re-check this behavior now that I've fixed the proxy schema to include both `memberOf` and `sAMAccountName`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KNOX-3277 : LDAP Proxy improvements for working with AD backend [knox]
Posted by "moresandeep (via GitHub)" <gi...@apache.org>.
moresandeep commented on code in PR #1177:
URL: https://github.com/apache/knox/pull/1177#discussion_r2960077228
##########
gateway-server/src/main/java/org/apache/knox/gateway/services/ldap/backend/LdapProxyBackend.java:
##########
@@ -63,6 +64,14 @@ public class LdapProxyBackend implements LdapBackend {
private String groupMemberAttribute = "memberUid"; // member for AD, memberUid for POSIX
private boolean useMemberOf; // Use memberOf attribute for group lookup (efficient for AD)
+ private List<String> proxyEntityAttributeTypes = List.of(
+ // "uid" will always be filled
+ "cn",
+ "dn",
Review Comment:
Thanks @handavid !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KNOX-3277 : LDAP Proxy improvements for working with AD backend [knox]
Posted by "handavid (via GitHub)" <gi...@apache.org>.
handavid commented on PR #1177:
URL: https://github.com/apache/knox/pull/1177#issuecomment-4098879633
> Thanks, @handavid, for your contribution. May I ask you to provide some information about the manual testing you did which includes the topology and the relevant settings? This would help us in the future to see the setup if we needed configure this feature for our own tests. Thanks!
@smolnar82
Here's a snipped of the gateway-site.xml that configures the LDAP Proxy and Backend. I've included configs for either the knox test ldap server or some AD server.
```
<!-- LDAP Proxy Service Configuration -->
<property>
<name>gateway.ldap.enabled</name>
<value>true</value>
<description>Enable the embedded LDAP service for user and group lookups. Set to true to enable.</description>
</property>
<property>
<name>gateway.ldap.port</name>
<value>3890</value>
<description>Port for the LDAP service to listen on. Default is 3890.</description>
</property>
<property>
<name>gateway.ldap.base.dn</name>
<value>dc=proxy,dc=com</value>
<description>Base DN for LDAP entries in the proxy server. Default is dc=proxy,dc=com.</description>
</property>
<property>
<name>gateway.ldap.backend.type</name>
<value>ldap</value>
<description>Backend type for LDAP service. Currently supported: file, ldap. Future: jdbc, knox.</description>
</property>
<!-- LDAP backend proxy configuration using knox test LDAP server -->
<!--
<property>
<name>gateway.ldap.backend.ldap.url</name>
<value>ldap://localhost:33389</value>
<description>LDAP server URL for proxy backend</description>
</property>
<property>
<name>gateway.ldap.backend.ldap.remoteBaseDn</name>
<value>dc=hadoop,dc=apache,dc=org</value>
<description>Base DN of the remote LDAP server</description>
</property>
<property>
<name>gateway.ldap.backend.ldap.systemUsername</name>
<value>uid=guest,ou=people,dc=hadoop,dc=apache,dc=org</value>
<description>LDAP bind DN for proxy backend authentication</description>
</property>
<property>
<name>gateway.ldap.backend.ldap.systemPassword</name>
<value>guest-password</value>
<description>LDAP bind password for proxy backend authentication</description>
</property>
<property>
<name>gateway.ldap.backend.data.file</name>
<value>${GATEWAY_DATA_HOME}/ldap-users.json</value>
<description>Path to JSON data file for file-based backend. Supports ${GATEWAY_DATA_HOME} variable.</description>
</property>
-->
<!-- LDAP backend proxy configuration using AD -->
<property>
<name>gateway.ldap.backend.ldap.url</name>
<value>ldap://<AD host>:389</value>
<description>LDAP server URL for proxy backend</description>
</property>
<property>
<name>gateway.ldap.backend.ldap.remoteBaseDn</name>
<value><REMOTE BASE DN></value>
<description>Base DN of the remote LDAP server</description>
</property>
<property>
<name>gateway.ldap.backend.ldap.systemUsername</name>
<value>cn=<REMOTE USERNAME>,cn=Users,<REMOTE BASE DN></value>
<description>LDAP bind DN for proxy backend authentication</description>
</property>
<property>
<name>gateway.ldap.backend.ldap.systemPassword</name>
<value><REMOTE PASSWORD></value>
<description>LDAP bind password for proxy backend authentication</description>
</property>
<property>
<name>gateway.ldap.backend.ldap.userIdentifierAttribute</name>
<!-- configure attribute to be used for user search against backend. Usually uid, cn, or sAMAccountName -->
<!-- <value>cn</value> -->
<value>sAMAccountName</value>
<description>Attribute used for identifying users</description>
</property>
<property>
<name>gateway.ldap.backend.ldap.userSearchBase</name>
<value>cn=users,<REMOTE BASE DN></value>
<description>Search base for users</description>
</property>
<property>
<name>gateway.ldap.backend.ldap.groupSearchBase</name>
<value>ou=groups,<REMOTE BASE DN></value>
<description>Search base for groups</description>
</property>
<property>
<name>gateway.ldap.backend.ldap.useMemberOf</name>
<value>true</value>
<description>Whether to use the memberOf attribute for efficiency when retrieving group memberships</description>
</property>
```
After building knox, I ran the following commands to install and start the test gateway
```
ant install-test-home
# update the gateway-site.xml with the above configs
ant start-test-gateway
```
I then tested using `ldapsearch` against the LDAP proxy and verified that the returned information matched the queries against AD
```
# single user search
ldapsearch -v -x -H ldap://localhost:3890 -b '<REMOTE BASE DN>' '(sAMAccountName=username)'
ldapsearch -v -x -H ldap://localhost:3890 -b '<REMOTE BASE DN>' '(cn=username)'
# user substring search
ldapsearch -v -x -H ldap://localhost:3890 -b '<REMOTE BASE DN>' '(sAMAccountName=substring*)'
ldapsearch -v -x -H ldap://localhost:3890 -b '<REMOTE BASE DN>' '(cn=substring*)'
ldapsearch -v -x -H ldap://localhost:3890 -b '<REMOTE BASE DN>' '(sAMAccountName=*)'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KNOX-3277 : LDAP Proxy improvements for working with AD backend [knox]
Posted by "handavid (via GitHub)" <gi...@apache.org>.
handavid commented on code in PR #1177:
URL: https://github.com/apache/knox/pull/1177#discussion_r2954864171
##########
gateway-server/src/main/java/org/apache/knox/gateway/services/ldap/GroupLookupInterceptor.java:
##########
@@ -44,6 +44,7 @@ public class GroupLookupInterceptor extends BaseInterceptor {
private LdapBackend backend;
private static final Pattern UID_PATTERN = Pattern.compile(".*\\(uid=([^)]+)\\).*");
private static final Pattern CN_PATTERN = Pattern.compile(".*\\(cn=([^)]+)\\).*");
+ private static final Pattern SAMAACCOUNTNAME_PATTERN = Pattern.compile(".*\\(sAMAccountName=([^)]+)\\).*");
Review Comment:
There's a name normalizing filter somewhere in the interceptor chain prior to this point. The filter will have the normalized `sAMAccountName` string.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KNOX-3277 : LDAP Proxy improvements for working with AD backend [knox]
Posted by "handavid (via GitHub)" <gi...@apache.org>.
handavid commented on code in PR #1177:
URL: https://github.com/apache/knox/pull/1177#discussion_r2955109101
##########
gateway-server/src/main/java/org/apache/knox/gateway/services/ldap/backend/LdapProxyBackend.java:
##########
@@ -63,6 +64,14 @@ public class LdapProxyBackend implements LdapBackend {
private String groupMemberAttribute = "memberUid"; // member for AD, memberUid for POSIX
private boolean useMemberOf; // Use memberOf attribute for group lookup (efficient for AD)
+ private List<String> proxyEntityAttributeTypes = List.of(
+ // "uid" will always be filled
+ "cn",
+ "dn",
Review Comment:
fixed naming
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KNOX-3277 : LDAP Proxy improvements for working with AD backend [knox]
Posted by "smolnar82 (via GitHub)" <gi...@apache.org>.
smolnar82 commented on PR #1177:
URL: https://github.com/apache/knox/pull/1177#issuecomment-4090608644
Thanks, @handavid, for your contribution.
May I ask you to provide some information about the manual testing you did which includes the topology and the relevant settings? This would help us in the future to see the setup if we needed configure this feature for our own tests.
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KNOX-3277. LDAP Proxy improvements for working with AD backend [knox]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #1177:
URL: https://github.com/apache/knox/pull/1177#issuecomment-4057374436
## Test Results
7 tests 7 ✅ 1s ⏱️
1 suites 0 💤
1 files 0 ❌
Results for commit 1c9e356e.
[test-results]:data:application/gzip;base64,H4sIAHdgtGkC/1WMQQ6DIBBFr2JYd6GgtNPLNDAOyaQqDcKq6d2LVi3u/ns/eW/heKBZ3KvmUok5cTygT8FE9tOG+YjLdd33Y06IZ/HkVxb1IZzh4SQoBB82E9K095ZZ5n78r61cxFYuW+jHkWMG0SCQ6jRJ0u0NesAWAJU0VpJSNUlrndIdOPH5AnpBzK/7AAAA
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KNOX-3277 : LDAP Proxy improvements for working with AD backend [knox]
Posted by "smolnar82 (via GitHub)" <gi...@apache.org>.
smolnar82 commented on PR #1177:
URL: https://github.com/apache/knox/pull/1177#issuecomment-4098983407
Thanks, @handavid, this is very useful!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KNOX-3277 : LDAP Proxy improvements for working with AD backend [knox]
Posted by "smolnar82 (via GitHub)" <gi...@apache.org>.
smolnar82 merged PR #1177:
URL: https://github.com/apache/knox/pull/1177
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org