KNOX-3330: Refactor LDAP Proxy configuration to support multiple backends by handavid · Pull Request #1240 · apache/knox

handavid · 2026-05-26T17:30:22Z

KNOX-3330 - Refactor Knox LDAP Proxy configuration and implementation to allow multiple backends to be simultaneously configured

What changes were proposed in this pull request?

Gateway server configurations are updated to use 'gateway.ldap.interceptor.' instead of 'gateway.ldap.backend.' to allow specifying multiple types of interceptors as well as multiple backends to the LDAP proxy.

BackendFactory has been modified to use the java ServiceLoader to load a factory for a backend class instead of a backend instance directly. This allows multiple backends of the same class to be configured. InterceptorFactory has been implemented following the same pattern.

GroupLookupInterceptor is renamed to UserSearchInterceptor to more accurately describe what it does. Multiple UserSearchInterceptors can be configured with each forwarding the search to its backend and appending the results.

A DuplicateUserFilteringInterceptor has been implemented that will filter out search Entries with the same UID that are returned from different backends.

How was this patch tested?

Unit tests were updated.

KnoxLDAPServerManagerTest.java modified to configure interceptors instead of backends
KnoxLDAPServerManagerTest.java modified to configure multiple backends simultaneously

Changes were manually tested against the test ldap server and an AD that I have access to.
The following configuration was added to the gateway-site.xml

    <!-- LDAP Proxy Service Configuration -->
    <property>
        <name>gateway.ldap.enabled</name>
        <value>true</value>
        <description>Enable the embedded LDAP service for user and group lookups. Set to true to enable.</description>
    </property>
    <property>
        <name>gateway.ldap.port</name>
        <value>3890</value>
        <description>Port for the LDAP service to listen on. Default is 3890.</description>
    </property>
    <property>
        <name>gateway.ldap.base.dn</name>
        <value>dc=proxy,dc=com</value>
        <description>Base DN for LDAP entries in the proxy server. Default is dc=proxy,dc=com.</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.names</name>
        <value>localldap,testad,duplicatefilter</value>
        <description>Interceptor names for LDAP service.</description>
    </property>

    <!-- Local LDAP Server -->
    <property>
        <name>gateway.ldap.interceptor.localldap.interceptorType</name>
        <value>backend</value>
        <description>Type of interceptor. Currently supported: backend, duplicateuserfilter</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.localldap.backendType</name>
        <value>ldap</value>
        <description>Type of backend. Currently supported: file, ldap. Future: jdbc, knox.</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.localldap.url</name>
        <value>ldap://localhost:33389</value>
        <description>LDAP server URL for proxy backend</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.localldap.remoteBaseDn</name>
        <value>dc=hadoop,dc=apache,dc=org</value>
        <description>Base DN of the remote LDAP server</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.localldap.systemUsername</name>
        <value>uid=guest,ou=people,dc=hadoop,dc=apache,dc=org</value>
        <description>LDAP bind DN for proxy backend authentication</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.localldap.systemPassword</name>
        <value>guest-password</value>
        <description>LDAP bind password for proxy backend authentication</description>
    </property>
    
    <!-- Test AD -->
    <property>
        <name>gateway.ldap.interceptor.testad.interceptorType</name>
        <value>backend</value>
        <description>Type of interceptor. Currently supported: backend, duplicateuserfilter</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.backendType</name>
        <value>ldap</value>
        <description>Type of backend. Currently supported: file, ldap. Future: jdbc, knox.</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.url</name>
        <value>ldap://test-ad.example.com:389</value>
        <description>LDAP server URL for proxy backend</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.remoteBaseDn</name>
        <value>dc=test-ad,dc=example,dc=com</value>
        <description>Base DN of the remote LDAP server</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.systemUsername</name>
        <value>bind user to AD</value>
        <description>LDAP bind DN for proxy backend authentication</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.systemPassword</name>
        <value>password to AD</value>
        <description>LDAP bind password for proxy backend authentication</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.userIdentifierAttribute</name>
        <value>sAMAccountName</value>
        <description>Attribute used for identifying users</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.userSearchBase</name>
        <value>cn=users,dc=test-ad,dc=example,dc=com</value>
        <description>Search base for users</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.groupSearchBase</name>
        <value>ou=groups,dc=test-ad,dc=example,dc=com</value>
        <description>Search base for groups</description>
    </property>
    <property>
        <name>gateway.ldap.interceptor.testad.useMemberOf</name>
        <value>true</value>
        <description>Whether to use the memberOf attribute for efficiency when retrieving group memberships</description>
    </property>

    <!-- Duplicate Filter Interceptor -->
    <property>
        <name>gateway.ldap.interceptor.duplicatefilter.interceptorType</name>
        <value>duplicateuserfilter</value>
        <description>Type of interceptor. Currently supported: backend, duplicateuserfilter</description>
    </property>

    <!-- END LDAP Proxy Service Configuration -->

Integration Tests

No integration test changes. PR can be updated after #1236 is merged

UI changes

no UI changes

handavid · 2026-05-26T17:31:08Z

@smolnar82 @lmccay

github-actions · 2026-05-27T14:48:56Z

Test Results

22 tests 22 ✅ 2s ⏱️
1 suites 0 💤
1 files 0 ❌

Results for commit 8e9af3d.

♻️ This comment has been updated with latest results.

handavid · 2026-06-02T05:54:25Z

rebased and fixed conflicts

smolnar82

I'll keep the review later today.

smolnar82

Another batch of comments.
I'll review the tests tomorrow.

smolnar82 · 2026-06-02T19:59:13Z

+    <!-- LDAP proxy backend configuration (gateway.ldap.interceptor.<interceptorName>.backendType=ldap) -->
    <!-- This backend proxies to an external LDAP server (e.g., demo LDAP) -->
    <!--
    Example 1: Using Knox demo LDAP server (default port 33389)


I'm not sure I like the idea of putting all these samples in the gateway-site.xml here.
We should rather create the new section in our user guide (see the knox-site module), and add all these sample configs there.

I'm fixing the existing configs here for my config changes. should we remove the whole ldap proxy-related block?
I was waiting for #1227 to be merged before updating the documentation

these values need to be correct for the KnoxCLITest to work

handavid · 2026-06-04T04:26:25Z

force push was due to rebase on master to work on resolution of conflicts.

…ends Gateway server configurations are updated to use 'gateway.ldap.interceptor.*' instead of 'gateway.ldap.backend.*' to allow specifying multiple types of interceptors as well as multiple backends to the LDAP proxy. BackendFactory has been modified to use the java ServiceLoader to load a factory for a backend class instead of a backend instance directly. This allows multiple backends of the same class to be configured. InterceptorFactory has been implemented following the same pattern. GroupLookupInterceptor is renamed to UserSearchInterceptor to more accurately describe what it does. Multiple UserSearchInterceptors can be configured with each forwarding the search to its backend and appending the results. A DuplicateUserFilteringInterceptor has been implemented that will filter out search Entries with the same UID that are returned from different backends.

handavid · 2026-06-04T22:11:14Z

@smolnar82
force pushed after rebase to pick up documentation changes.
I've updated the service_ldap_server.md to reflect the architectural and configuration changes.
I've also fixed the KnoxCLI to start the KnoxLDAPService. This is needed to start the directoryService. I tested this by running:

$ ./knoxcli.sh ldap-user-groups-test --u sam
Querying KnoxLDAPService for groups of user: sam
sam is a member of: scientist, analyst

handavid force-pushed the knox-3330-improve-interceptors branch from 33fe1fe to dd9de8d Compare June 2, 2026 05:53

smolnar82 requested changes Jun 2, 2026

View reviewed changes

handavid force-pushed the knox-3330-improve-interceptors branch from 0bfb637 to 69960ee Compare June 4, 2026 04:25

handavid requested a review from smolnar82 June 4, 2026 04:29

handavid added 2 commits June 4, 2026 15:44

fix KnoxCLI to start KnoxLDAPService. update docs

423e3ae

handavid force-pushed the knox-3330-improve-interceptors branch from 8e9af3d to 423e3ae Compare June 4, 2026 22:08

Conversation

handavid commented May 26, 2026

What changes were proposed in this pull request?

How was this patch tested?

Integration Tests

UI changes

Uh oh!

handavid commented May 26, 2026

Uh oh!

github-actions Bot commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

handavid commented Jun 2, 2026

Uh oh!

smolnar82 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

smolnar82 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

smolnar82 Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

handavid Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

handavid Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

handavid commented Jun 4, 2026

Uh oh!

handavid commented Jun 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented May 27, 2026 •

edited

Loading