You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ranger.apache.org by ra...@apache.org on 2026/06/14 17:13:04 UTC

(ranger) 01/01: RANGER-5645: Add audit-ingestor service-user allowlist for Docker plugins

This is an automated email from the ASF dual-hosted git repository.

ramackri pushed a commit to branch RANGER-5645-patch
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit d05e556380eca8a56bad33de97344e45c533135b
Author: ramk <ra...@cloudera.com>
AuthorDate: Sun Jun 14 22:42:50 2026 +0530

    RANGER-5645: Add audit-ingestor service-user allowlist for Docker plugins
    
    Ship per-repo allowed.users and auth_to_local rules so plugins using the
    audit-server destination are authorized after Kerberos SPNEGO (fixes HTTP 403).
    Align create-ranger-services.py with policy.download.auth.users for Ozone,
    Atlas, Kudu, and NiFi. Add troubleshooting README for ingestor 403 errors.
    
    Co-authored-by: Cursor <cu...@cursor.com>
---
 .../README-AUDIT-INGESTOR-SERVICE-ALLOWLIST.md     | 197 +++++++++++++++++++++
 .../resources/conf/ranger-audit-ingestor-site.xml  | 107 +++++++++--
 audit-server/scripts/README.md                     |   2 +
 .../scripts/admin/create-ranger-services.py        |  33 +++-
 4 files changed, 320 insertions(+), 19 deletions(-)

diff --git a/audit-server/README-AUDIT-INGESTOR-SERVICE-ALLOWLIST.md b/audit-server/README-AUDIT-INGESTOR-SERVICE-ALLOWLIST.md
new file mode 100644
index 000000000..7093b044a
--- /dev/null
+++ b/audit-server/README-AUDIT-INGESTOR-SERVICE-ALLOWLIST.md
@@ -0,0 +1,197 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+  http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Audit ingestor: plugin 403 (service allowlist)
+
+When a Ranger plugin uses the **audit-server** destination (`xasecure.audit.destination.auditserver`), it POSTs audit batches to audit-ingestor at `/audit/access?serviceName=<repo>`. Failures often look like authentication problems but are usually **authorization (403)** — Kerberos/SPNEGO succeeded; the mapped service user is not on the ingestor allowlist for that Policy Manager repo.
+
+Related: [dev-support/ranger-docker/README.md](../dev-support/ranger-docker/README.md), shipped defaults in `audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml`.
+
+---
+
+## Quick read: 401 vs 403
+
+| HTTP | Meaning | Typical cause |
+|------|---------|---------------|
+| **401** | Authentication failed | Missing/invalid Kerberos ticket, wrong SPNEGO, clock skew, wrong ingestor URL |
+| **403** | Authentication OK, **authorization failed** | Caller authenticated but short username not in `ranger.audit.ingestor.service.<repo>.allowed.users` |
+
+If ingestor logs show `Unauthorized user: user=<name> ... service=<repo>`, Kerberos worked — fix the allowlist or `auth_to_local`, not the keytab.
+
+---
+
+## What's happening
+
+```text
+Plugin (e.g. KMS)                    audit-ingestor
+     |                                      |
+     |  POST /audit/access?serviceName=dev_kms
+     |  Authorization: Negotiate (SPNEGO)   |
+     |------------------------------------->|
+     |                                      | 1. Authenticate Kerberos principal
+     |                                      |    e.g. rangerkms/host@REALM
+     |                                      | 2. auth_to_local → short name "rangerkms"
+     |                                      | 3. isAllowedServiceUser(dev_kms, rangerkms)
+     |                                      |    checks allowed.users for dev_kms
+     |                                      | 4. Not listed → 403 Forbidden
+     |<-------------------------------------|
+     |  {"message":"User is not authorized to send audit events",...}
+```
+
+Implementation: `AuditREST.java` → `isAllowedServiceUser()`. Config keys:
+
+```properties
+ranger.audit.ingestor.service.<repoName>.allowed.users=<comma-separated short names>
+ranger.audit.ingestor.auth.to.local=<Kerberos RULE lines>
+```
+
+`<repoName>` must match the Policy Manager **service name** (e.g. `dev_kms`, `dev_hdfs`), not the plugin type alone.
+
+---
+
+## Example: Ranger KMS (403)
+
+**Plugin log:**
+
+```text
+ERROR o.a.r.a.d.RangerAuditServerDestination - Failed to send audit batch.
+HTTP status: 403, Response: {"message":"User is not authorized to send audit events",...}
+ERROR o.a.r.a.d.RangerAuditServerDestination - Failed to send batch of 1 events
+```
+
+**audit-ingestor log:**
+
+```text
+ERROR - Unauthorized user: user=rangerkms is authorized report audit logs for service=dev_kms.
+Rejecting audit request.
+```
+
+**Interpretation:** SPNEGO to ingestor works. `rangerkms` is not allowed for repo `dev_kms` (missing property, wrong short name, or stale runtime config).
+
+**Fix:**
+
+```xml
+<property>
+    <name>ranger.audit.ingestor.service.dev_kms.allowed.users</name>
+    <value>rangerkms</value>
+</property>
+```
+
+Ensure `auth_to_local` includes `rangerkms/*` → `rangerkms`, restart ingestor, confirm the **running** pod/container loads the updated site XML.
+
+---
+
+## Root causes (side-by-side)
+
+| # | Symptom | Likely cause | Fix |
+|---|---------|--------------|-----|
+| 1 | 403, ingestor log names user + repo | No `allowed.users` for that repo | Add `ranger.audit.ingestor.service.<repo>.allowed.users` |
+| 2 | 403, user name looks wrong (e.g. `kms` vs `rangerkms`) | Allowlist short name ≠ `auth_to_local` output | Align with `policy.download.auth.users` in Ranger Admin |
+| 3 | 403 for Ozone OM/SCM/DN only | Allowlist has `ozone` but principal maps to `om`/`scm`/`dn` | Use `ozone,om,scm,dn` for `dev_ozone` |
+| 4 | 403 after editing source XML only | Ingestor still using old classpath copy | Patch `conf/` **and** `WEB-INF/classes/conf/`, or rebuild/redeploy image |
+| 5 | 403, repo name unexpected | Plugin `serviceName` ≠ Policy Manager service name | Match repo name in plugin audit config to Ranger Admin service |
+| 6 | 401 instead of 403 | Kerberos/SPNEGO failure | Keytab, principal, `_HOST`, ingestor URL/FQDN, KDC — not allowlist |
+
+---
+
+## How to fix
+
+### 1. Identify repo and short username
+
+From ingestor error line: `user=<shortname> ... service=<repo>`.
+
+Or from Ranger Admin → service → **Configs** → `policy.download.auth.users` (ingestor allowlist should match).
+
+### 2. Add or update allowlist
+
+In `ranger-audit-ingestor-site.xml` (or deployment overlay):
+
+```xml
+<property>
+    <name>ranger.audit.ingestor.service.dev_<service>.allowed.users</name>
+    <value><short names, comma-separated></value>
+</property>
+```
+
+### 3. Verify `auth_to_local`
+
+Property: `ranger.audit.ingestor.auth.to_local`. Service-specific `RULE:` lines must run **before** the generic `user@REALM` fallback so `service/host@REALM` maps to the same short names as the allowlist.
+
+### 4. Restart audit-ingestor
+
+Allowlist is loaded at startup. After change, restart the ingestor pod/container.
+
+### 5. Docker dev stack extras
+
+| Step | Action |
+|------|--------|
+| Policy Manager repos | `docker exec ranger python3 /home/ranger/scripts/create-ranger-services.py` (idempotent) |
+| New plugin / new repo | Add allowlist entry + `auth_to_local` rule + create service in Admin |
+
+---
+
+## Shipped allowlist reference (Docker dev repos)
+
+Align each row with `policy.download.auth.users` for that service in Policy Manager.
+
+| Policy Manager repo | `allowed.users` (short names) |
+|---------------------|-------------------------------|
+| `dev_hdfs` | `hdfs` |
+| `dev_yarn` | `yarn` |
+| `dev_hive` | `hive` |
+| `dev_hbase` | `hbase` |
+| `dev_kafka` | `kafka` |
+| `dev_knox` | `knox` |
+| `dev_kms` | `rangerkms` |
+| `dev_trino` | `trino` |
+| `dev_ozone` | `ozone,om,scm,dn` |
+| `dev_solr` | `solr` |
+| `dev_atlas` | `atlas` |
+| `dev_kudu` | `kudu` |
+| `dev_nifi` | `nifi` |
+
+For a **new** repo, add both the property and a matching `auth_to_local` rule; there is no wildcard allowlist.
+
+---
+
+## Verification
+
+1. Trigger an access event on the plugin (e.g. KMS operation).
+2. Plugin log: no `HTTP status: 403` from `RangerAuditServerDestination`.
+3. Ingestor log: no `Unauthorized user` for that repo; debug may show `isAllowedServiceUser(...): ret=true`.
+4. Downstream (optional): audit count increases in Kafka/Solr/Admin UI.
+
+---
+
+## Product vs Docker scope
+
+| Scope | Problem | Owner / fix |
+|-------|---------|-------------|
+| **Product** | Fresh install missing allowlist for enabled plugins | Ship complete defaults in `ranger-audit-ingestor-site.xml`; document in install guide |
+| **Docker / ops** | Stack built before allowlist was added; stale WEB-INF copy | Patch/redeploy ingestor; run `create-ranger-services.py` |
+
+Same 403 mechanism in both cases; difference is whether defaults are shipped vs patched at deploy time.
+
+---
+
+## Code references
+
+| Piece | Location |
+|-------|----------|
+| Authorization check | `audit-ingestor/.../AuditREST.java` — `isAllowedServiceUser()` |
+| Property prefix | `ranger.audit.ingestor.service.` + `<repo>` + `.allowed.users` |
+| Default config | `audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml` |
+| Plugin client | `agents-audit/.../RangerAuditServerDestination.java` |
diff --git a/audit-server/audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml b/audit-server/audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml
index 976c2a0cb..e75f70904 100644
--- a/audit-server/audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml
+++ b/audit-server/audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml
@@ -185,19 +185,79 @@
     <property>
         <name>ranger.audit.ingestor.service.dev_hdfs.allowed.users</name>
         <value>hdfs</value>
-        <description>Comma-separated list of allowed users that can send audits for dev_hdfs service</description>
+        <description>Allowed users for dev_hdfs (Policy Manager service name; from policy.download.auth.users)</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_yarn.allowed.users</name>
+        <value>yarn</value>
+        <description>Allowed users for dev_yarn (YARN RM/NM)</description>
     </property>
 
     <property>
         <name>ranger.audit.ingestor.service.dev_hive.allowed.users</name>
         <value>hive</value>
-        <description>Comma-separated list of allowed users that can send audits for dev_hive service.</description>
+        <description>Allowed users for dev_hive (HiveServer2 + Metastore)</description>
     </property>
 
     <property>
         <name>ranger.audit.ingestor.service.dev_hbase.allowed.users</name>
         <value>hbase</value>
-        <description>Comma-separated list of allowed users that can send audits for dev_hbase service.</description>
+        <description>Allowed users for dev_hbase (HBase Master + RegionServer)</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_kafka.allowed.users</name>
+        <value>kafka</value>
+        <description>Allowed users for dev_kafka</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_knox.allowed.users</name>
+        <value>knox</value>
+        <description>Allowed users for dev_knox</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_kms.allowed.users</name>
+        <value>rangerkms</value>
+        <description>Allowed users for dev_kms</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_trino.allowed.users</name>
+        <value>trino</value>
+        <description>Allowed users for dev_trino</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_ozone.allowed.users</name>
+        <value>ozone,om,scm,dn</value>
+        <description>Allowed users for dev_ozone (OM, SCM, DataNode)</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_solr.allowed.users</name>
+        <value>solr</value>
+        <description>Allowed users for dev_solr (Solr plugin)</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_atlas.allowed.users</name>
+        <value>atlas</value>
+        <description>Allowed users for dev_atlas (Atlas)</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_kudu.allowed.users</name>
+        <value>kudu</value>
+        <description>Allowed users for dev_kudu (Kudu master/tserver)</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_nifi.allowed.users</name>
+        <value>nifi</value>
+        <description>Allowed users for dev_nifi (NiFi)</description>
     </property>
 
     <!-- AUTH_TO_LOCAL RULES FOR KERBEROS PRINCIPAL MAPPING -->
@@ -207,22 +267,39 @@
                RULE:[2:$1/$2@$0]([ndj]n/.*@.*|hdfs/.*@.*)s/.*/hdfs/
                RULE:[2:$1/$2@$0]([rn]m/.*@.*|yarn/.*@.*)s/.*/yarn/
                RULE:[2:$1/$2@$0](jhs/.*@.*)s/.*/mapred/
+               RULE:[2:$1/$2@$0](hive/.*@.*)s/.*/hive/
+               RULE:[2:$1/$2@$0](hbase/.*@.*)s/.*/hbase/
+               RULE:[2:$1/$2@$0](kafka/.*@.*)s/.*/kafka/
+               RULE:[2:$1/$2@$0](knox/.*@.*)s/.*/knox/
+               RULE:[2:$1/$2@$0](rangerkms/.*@.*)s/.*/rangerkms/
+               RULE:[2:$1/$2@$0](trino/.*@.*)s/.*/trino/
+               RULE:[2:$1/$2@$0](ozone/.*@.*)s/.*/ozone/
+               RULE:[2:$1/$2@$0](om/.*@.*)s/.*/om/
+               RULE:[2:$1/$2@$0](scm/.*@.*)s/.*/scm/
+               RULE:[2:$1/$2@$0](dn/.*@.*)s/.*/dn/
+               RULE:[2:$1/$2@$0](solr/.*@.*)s/.*/solr/
+               RULE:[2:$1/$2@$0](atlas/.*@.*)s/.*/atlas/
+               RULE:[2:$1/$2@$0](kudu/.*@.*)s/.*/kudu/
+               RULE:[2:$1/$2@$0](nifi/.*@.*)s/.*/nifi/
                RULE:[1:$1@$0](.*@.*)s/@.*//
                DEFAULT
         </value>
         <description>
-            Kerberos auth_to_local rules for mapping authenticated principals to service names.
-            Uses Hadoop KerberosName syntax to convert full Kerberos principals to short usernames.
-
-            IMPORTANT: These rules only apply to Kerberos/SPNEGO authentication. JWT and basic auth
-            usernames are already in short form and pass through unchanged.
-
-            Default rules provided:
-            - RULE:[2:$1/$2@$0]([ndj]n/.*@.*|hdfs/.*@.*)s/.*/hdfs/     # nn,dn,jn,hdfs/* -> hdfs
-            - RULE:[2:$1/$2@$0]([rn]m/.*@.*|yarn/.*@.*)s/.*/yarn/      # rm,nm,yarn/* -> yarn
-            - RULE:[2:$1/$2@$0](jhs/.*@.*)s/.*/mapred/                 # jhs/* -> mapred
-            - RULE:[1:$1@$0](.*@.*)s/@.*//                             # user@REALM -> user
-            - DEFAULT
+            Kerberos auth_to_local rules: map service/host@REALM to the short username checked
+            against ranger.audit.ingestor.service.&lt;repo&gt;.allowed.users (Policy Manager
+            policy.download.auth.users).
+
+            Order matters — service-specific RULE lines run before the user@REALM fallback.
+
+            Hadoop: nn/dn/jn/hdfs/* -&gt; hdfs (dev_hdfs); rm/nm/yarn/* -&gt; yarn (dev_yarn);
+            jhs/* -&gt; mapred.
+            Plugins: hive/* -&gt; hive (dev_hive); hbase/* -&gt; hbase (dev_hbase);
+            kafka/* -&gt; kafka (dev_kafka); knox/* -&gt; knox (dev_knox);
+            rangerkms/* -&gt; rangerkms (dev_kms); trino/* -&gt; trino (dev_trino);
+            ozone/* -&gt; ozone; om/* -&gt; om; scm/* -&gt; scm; dn/* -&gt; dn (dev_ozone);
+            solr/* -&gt; solr (dev_solr); atlas/* -&gt; atlas (dev_atlas);
+            kudu/* -&gt; kudu (dev_kudu); nifi/* -&gt; nifi (dev_nifi).
+            Fallback: user@REALM -&gt; user; then DEFAULT.
         </description>
     </property>
 
diff --git a/audit-server/scripts/README.md b/audit-server/scripts/README.md
index 6bd09a03b..917b0b02e 100644
--- a/audit-server/scripts/README.md
+++ b/audit-server/scripts/README.md
@@ -30,6 +30,8 @@ The codebase is organized into two main directories:
 - `audit-ingestor/`
 - `audit-dispatcher/` (contains a unified dispatcher application that dynamically loads specific dispatcher types)
 
+**Troubleshooting:** Plugin audits rejected with HTTP 403 from ingestor (KMS, Kafka, Ozone, etc.) — see [README-AUDIT-INGESTOR-SERVICE-ALLOWLIST.md](../README-AUDIT-INGESTOR-SERVICE-ALLOWLIST.md).
+
 ## Prerequisites
 
 Before running these scripts, ensure you have:
diff --git a/dev-support/ranger-docker/scripts/admin/create-ranger-services.py b/dev-support/ranger-docker/scripts/admin/create-ranger-services.py
index 68d9b915e..6f85c2371 100644
--- a/dev-support/ranger-docker/scripts/admin/create-ranger-services.py
+++ b/dev-support/ranger-docker/scripts/admin/create-ranger-services.py
@@ -129,9 +129,9 @@ def service_not_exists(service):
                        'configs': {'username': 'hdfs', 'password': 'hdfs',
                                    'ozone.om.http-address': 'http://om:9874',
                                    'hadoop.security.authentication': 'simple',
-                                   'policy.download.auth.users': 'ozone',
-                                   'tag.download.auth.users': 'ozone',
-                                   'userstore.download.auth.users': 'ozone',
+                                   'policy.download.auth.users': 'ozone,om,scm,dn',
+                                   'tag.download.auth.users': 'ozone,om,scm,dn',
+                                   'userstore.download.auth.users': 'ozone,om,scm,dn',
                                    'ranger.plugin.ozone.policy.refresh.synchronous':'true'}})
 
 solr = RangerService({'name': 'dev_solr', 'type': 'solr',
@@ -148,7 +148,32 @@ def service_not_exists(service):
                                  'ranger.plugin.super.users': 'solr',
                                  'ranger.plugin.solr.policy.refresh.synchronous':'true'}})
 
-services = [hdfs, yarn, hive, hbase, kafka, knox, kms, trino, ozone, solr]
+atlas = RangerService({'name': 'dev_atlas', 'type': 'atlas',
+                       'configs': {'username': 'atlas', 'password': 'atlas',
+                                   'atlas.rest.address': 'http://atlas:21000',
+                                   'policy.download.auth.users': 'atlas',
+                                   'tag.download.auth.users': 'atlas',
+                                   'userstore.download.auth.users': 'atlas',
+                                   'ranger.plugin.atlas.policy.refresh.synchronous': 'true'}})
+
+kudu = RangerService({'name': 'dev_kudu', 'type': 'kudu',
+                      'configs': {'username': 'kudu', 'password': 'kudu',
+                                  'policy.download.auth.users': 'kudu',
+                                  'tag.download.auth.users': 'kudu',
+                                  'userstore.download.auth.users': 'kudu',
+                                  'ranger.plugin.kudu.policy.refresh.synchronous': 'true'}})
+
+nifi = RangerService({'name': 'dev_nifi', 'type': 'nifi',
+                      'configs': {'username': 'nifi', 'password': 'nifi',
+                                  'nifi.url': 'http://ranger-nifi.rangernw:8080/nifi-api/resources',
+                                  'nifi.authentication': 'NONE',
+                                  'nifi.ssl.use.default.context': 'false',
+                                  'policy.download.auth.users': 'nifi',
+                                  'tag.download.auth.users': 'nifi',
+                                  'userstore.download.auth.users': 'nifi',
+                                  'ranger.plugin.nifi.policy.refresh.synchronous': 'true'}})
+
+services = [hdfs, yarn, hive, hbase, kafka, knox, kms, trino, ozone, solr, atlas, kudu, nifi]
 for service in services:
     try:
         if service_not_exists(service):