You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ranger.apache.org by ra...@apache.org on 2026/06/14 17:13:04 UTC
(ranger) 01/01: RANGER-5645: Add audit-ingestor service-user allowlist for Docker plugins
This is an automated email from the ASF dual-hosted git repository.
ramackri pushed a commit to branch RANGER-5645-patch
in repository https://gitbox.apache.org/repos/asf/ranger.git
commit d05e556380eca8a56bad33de97344e45c533135b
Author: ramk <ra...@cloudera.com>
AuthorDate: Sun Jun 14 22:42:50 2026 +0530
RANGER-5645: Add audit-ingestor service-user allowlist for Docker plugins
Ship per-repo allowed.users and auth_to_local rules so plugins using the
audit-server destination are authorized after Kerberos SPNEGO (fixes HTTP 403).
Align create-ranger-services.py with policy.download.auth.users for Ozone,
Atlas, Kudu, and NiFi. Add troubleshooting README for ingestor 403 errors.
Co-authored-by: Cursor <cu...@cursor.com>
---
.../README-AUDIT-INGESTOR-SERVICE-ALLOWLIST.md | 197 +++++++++++++++++++++
.../resources/conf/ranger-audit-ingestor-site.xml | 107 +++++++++--
audit-server/scripts/README.md | 2 +
.../scripts/admin/create-ranger-services.py | 33 +++-
4 files changed, 320 insertions(+), 19 deletions(-)
diff --git a/audit-server/README-AUDIT-INGESTOR-SERVICE-ALLOWLIST.md b/audit-server/README-AUDIT-INGESTOR-SERVICE-ALLOWLIST.md
new file mode 100644
index 000000000..7093b044a
--- /dev/null
+++ b/audit-server/README-AUDIT-INGESTOR-SERVICE-ALLOWLIST.md
@@ -0,0 +1,197 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Audit ingestor: plugin 403 (service allowlist)
+
+When a Ranger plugin uses the **audit-server** destination (`xasecure.audit.destination.auditserver`), it POSTs audit batches to audit-ingestor at `/audit/access?serviceName=<repo>`. Failures often look like authentication problems but are usually **authorization (403)** — Kerberos/SPNEGO succeeded; the mapped service user is not on the ingestor allowlist for that Policy Manager repo.
+
+Related: [dev-support/ranger-docker/README.md](../dev-support/ranger-docker/README.md), shipped defaults in `audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml`.
+
+---
+
+## Quick read: 401 vs 403
+
+| HTTP | Meaning | Typical cause |
+|------|---------|---------------|
+| **401** | Authentication failed | Missing/invalid Kerberos ticket, wrong SPNEGO, clock skew, wrong ingestor URL |
+| **403** | Authentication OK, **authorization failed** | Caller authenticated but short username not in `ranger.audit.ingestor.service.<repo>.allowed.users` |
+
+If ingestor logs show `Unauthorized user: user=<name> ... service=<repo>`, Kerberos worked — fix the allowlist or `auth_to_local`, not the keytab.
+
+---
+
+## What's happening
+
+```text
+Plugin (e.g. KMS) audit-ingestor
+ | |
+ | POST /audit/access?serviceName=dev_kms
+ | Authorization: Negotiate (SPNEGO) |
+ |------------------------------------->|
+ | | 1. Authenticate Kerberos principal
+ | | e.g. rangerkms/host@REALM
+ | | 2. auth_to_local → short name "rangerkms"
+ | | 3. isAllowedServiceUser(dev_kms, rangerkms)
+ | | checks allowed.users for dev_kms
+ | | 4. Not listed → 403 Forbidden
+ |<-------------------------------------|
+ | {"message":"User is not authorized to send audit events",...}
+```
+
+Implementation: `AuditREST.java` → `isAllowedServiceUser()`. Config keys:
+
+```properties
+ranger.audit.ingestor.service.<repoName>.allowed.users=<comma-separated short names>
+ranger.audit.ingestor.auth.to.local=<Kerberos RULE lines>
+```
+
+`<repoName>` must match the Policy Manager **service name** (e.g. `dev_kms`, `dev_hdfs`), not the plugin type alone.
+
+---
+
+## Example: Ranger KMS (403)
+
+**Plugin log:**
+
+```text
+ERROR o.a.r.a.d.RangerAuditServerDestination - Failed to send audit batch.
+HTTP status: 403, Response: {"message":"User is not authorized to send audit events",...}
+ERROR o.a.r.a.d.RangerAuditServerDestination - Failed to send batch of 1 events
+```
+
+**audit-ingestor log:**
+
+```text
+ERROR - Unauthorized user: user=rangerkms is authorized report audit logs for service=dev_kms.
+Rejecting audit request.
+```
+
+**Interpretation:** SPNEGO to ingestor works. `rangerkms` is not allowed for repo `dev_kms` (missing property, wrong short name, or stale runtime config).
+
+**Fix:**
+
+```xml
+<property>
+ <name>ranger.audit.ingestor.service.dev_kms.allowed.users</name>
+ <value>rangerkms</value>
+</property>
+```
+
+Ensure `auth_to_local` includes `rangerkms/*` → `rangerkms`, restart ingestor, confirm the **running** pod/container loads the updated site XML.
+
+---
+
+## Root causes (side-by-side)
+
+| # | Symptom | Likely cause | Fix |
+|---|---------|--------------|-----|
+| 1 | 403, ingestor log names user + repo | No `allowed.users` for that repo | Add `ranger.audit.ingestor.service.<repo>.allowed.users` |
+| 2 | 403, user name looks wrong (e.g. `kms` vs `rangerkms`) | Allowlist short name ≠ `auth_to_local` output | Align with `policy.download.auth.users` in Ranger Admin |
+| 3 | 403 for Ozone OM/SCM/DN only | Allowlist has `ozone` but principal maps to `om`/`scm`/`dn` | Use `ozone,om,scm,dn` for `dev_ozone` |
+| 4 | 403 after editing source XML only | Ingestor still using old classpath copy | Patch `conf/` **and** `WEB-INF/classes/conf/`, or rebuild/redeploy image |
+| 5 | 403, repo name unexpected | Plugin `serviceName` ≠ Policy Manager service name | Match repo name in plugin audit config to Ranger Admin service |
+| 6 | 401 instead of 403 | Kerberos/SPNEGO failure | Keytab, principal, `_HOST`, ingestor URL/FQDN, KDC — not allowlist |
+
+---
+
+## How to fix
+
+### 1. Identify repo and short username
+
+From ingestor error line: `user=<shortname> ... service=<repo>`.
+
+Or from Ranger Admin → service → **Configs** → `policy.download.auth.users` (ingestor allowlist should match).
+
+### 2. Add or update allowlist
+
+In `ranger-audit-ingestor-site.xml` (or deployment overlay):
+
+```xml
+<property>
+ <name>ranger.audit.ingestor.service.dev_<service>.allowed.users</name>
+ <value><short names, comma-separated></value>
+</property>
+```
+
+### 3. Verify `auth_to_local`
+
+Property: `ranger.audit.ingestor.auth.to_local`. Service-specific `RULE:` lines must run **before** the generic `user@REALM` fallback so `service/host@REALM` maps to the same short names as the allowlist.
+
+### 4. Restart audit-ingestor
+
+Allowlist is loaded at startup. After change, restart the ingestor pod/container.
+
+### 5. Docker dev stack extras
+
+| Step | Action |
+|------|--------|
+| Policy Manager repos | `docker exec ranger python3 /home/ranger/scripts/create-ranger-services.py` (idempotent) |
+| New plugin / new repo | Add allowlist entry + `auth_to_local` rule + create service in Admin |
+
+---
+
+## Shipped allowlist reference (Docker dev repos)
+
+Align each row with `policy.download.auth.users` for that service in Policy Manager.
+
+| Policy Manager repo | `allowed.users` (short names) |
+|---------------------|-------------------------------|
+| `dev_hdfs` | `hdfs` |
+| `dev_yarn` | `yarn` |
+| `dev_hive` | `hive` |
+| `dev_hbase` | `hbase` |
+| `dev_kafka` | `kafka` |
+| `dev_knox` | `knox` |
+| `dev_kms` | `rangerkms` |
+| `dev_trino` | `trino` |
+| `dev_ozone` | `ozone,om,scm,dn` |
+| `dev_solr` | `solr` |
+| `dev_atlas` | `atlas` |
+| `dev_kudu` | `kudu` |
+| `dev_nifi` | `nifi` |
+
+For a **new** repo, add both the property and a matching `auth_to_local` rule; there is no wildcard allowlist.
+
+---
+
+## Verification
+
+1. Trigger an access event on the plugin (e.g. KMS operation).
+2. Plugin log: no `HTTP status: 403` from `RangerAuditServerDestination`.
+3. Ingestor log: no `Unauthorized user` for that repo; debug may show `isAllowedServiceUser(...): ret=true`.
+4. Downstream (optional): audit count increases in Kafka/Solr/Admin UI.
+
+---
+
+## Product vs Docker scope
+
+| Scope | Problem | Owner / fix |
+|-------|---------|-------------|
+| **Product** | Fresh install missing allowlist for enabled plugins | Ship complete defaults in `ranger-audit-ingestor-site.xml`; document in install guide |
+| **Docker / ops** | Stack built before allowlist was added; stale WEB-INF copy | Patch/redeploy ingestor; run `create-ranger-services.py` |
+
+Same 403 mechanism in both cases; difference is whether defaults are shipped vs patched at deploy time.
+
+---
+
+## Code references
+
+| Piece | Location |
+|-------|----------|
+| Authorization check | `audit-ingestor/.../AuditREST.java` — `isAllowedServiceUser()` |
+| Property prefix | `ranger.audit.ingestor.service.` + `<repo>` + `.allowed.users` |
+| Default config | `audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml` |
+| Plugin client | `agents-audit/.../RangerAuditServerDestination.java` |
diff --git a/audit-server/audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml b/audit-server/audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml
index 976c2a0cb..e75f70904 100644
--- a/audit-server/audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml
+++ b/audit-server/audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml
@@ -185,19 +185,79 @@
<property>
<name>ranger.audit.ingestor.service.dev_hdfs.allowed.users</name>
<value>hdfs</value>
- <description>Comma-separated list of allowed users that can send audits for dev_hdfs service</description>
+ <description>Allowed users for dev_hdfs (Policy Manager service name; from policy.download.auth.users)</description>
+ </property>
+
+ <property>
+ <name>ranger.audit.ingestor.service.dev_yarn.allowed.users</name>
+ <value>yarn</value>
+ <description>Allowed users for dev_yarn (YARN RM/NM)</description>
</property>
<property>
<name>ranger.audit.ingestor.service.dev_hive.allowed.users</name>
<value>hive</value>
- <description>Comma-separated list of allowed users that can send audits for dev_hive service.</description>
+ <description>Allowed users for dev_hive (HiveServer2 + Metastore)</description>
</property>
<property>
<name>ranger.audit.ingestor.service.dev_hbase.allowed.users</name>
<value>hbase</value>
- <description>Comma-separated list of allowed users that can send audits for dev_hbase service.</description>
+ <description>Allowed users for dev_hbase (HBase Master + RegionServer)</description>
+ </property>
+
+ <property>
+ <name>ranger.audit.ingestor.service.dev_kafka.allowed.users</name>
+ <value>kafka</value>
+ <description>Allowed users for dev_kafka</description>
+ </property>
+
+ <property>
+ <name>ranger.audit.ingestor.service.dev_knox.allowed.users</name>
+ <value>knox</value>
+ <description>Allowed users for dev_knox</description>
+ </property>
+
+ <property>
+ <name>ranger.audit.ingestor.service.dev_kms.allowed.users</name>
+ <value>rangerkms</value>
+ <description>Allowed users for dev_kms</description>
+ </property>
+
+ <property>
+ <name>ranger.audit.ingestor.service.dev_trino.allowed.users</name>
+ <value>trino</value>
+ <description>Allowed users for dev_trino</description>
+ </property>
+
+ <property>
+ <name>ranger.audit.ingestor.service.dev_ozone.allowed.users</name>
+ <value>ozone,om,scm,dn</value>
+ <description>Allowed users for dev_ozone (OM, SCM, DataNode)</description>
+ </property>
+
+ <property>
+ <name>ranger.audit.ingestor.service.dev_solr.allowed.users</name>
+ <value>solr</value>
+ <description>Allowed users for dev_solr (Solr plugin)</description>
+ </property>
+
+ <property>
+ <name>ranger.audit.ingestor.service.dev_atlas.allowed.users</name>
+ <value>atlas</value>
+ <description>Allowed users for dev_atlas (Atlas)</description>
+ </property>
+
+ <property>
+ <name>ranger.audit.ingestor.service.dev_kudu.allowed.users</name>
+ <value>kudu</value>
+ <description>Allowed users for dev_kudu (Kudu master/tserver)</description>
+ </property>
+
+ <property>
+ <name>ranger.audit.ingestor.service.dev_nifi.allowed.users</name>
+ <value>nifi</value>
+ <description>Allowed users for dev_nifi (NiFi)</description>
</property>
<!-- AUTH_TO_LOCAL RULES FOR KERBEROS PRINCIPAL MAPPING -->
@@ -207,22 +267,39 @@
RULE:[2:$1/$2@$0]([ndj]n/.*@.*|hdfs/.*@.*)s/.*/hdfs/
RULE:[2:$1/$2@$0]([rn]m/.*@.*|yarn/.*@.*)s/.*/yarn/
RULE:[2:$1/$2@$0](jhs/.*@.*)s/.*/mapred/
+ RULE:[2:$1/$2@$0](hive/.*@.*)s/.*/hive/
+ RULE:[2:$1/$2@$0](hbase/.*@.*)s/.*/hbase/
+ RULE:[2:$1/$2@$0](kafka/.*@.*)s/.*/kafka/
+ RULE:[2:$1/$2@$0](knox/.*@.*)s/.*/knox/
+ RULE:[2:$1/$2@$0](rangerkms/.*@.*)s/.*/rangerkms/
+ RULE:[2:$1/$2@$0](trino/.*@.*)s/.*/trino/
+ RULE:[2:$1/$2@$0](ozone/.*@.*)s/.*/ozone/
+ RULE:[2:$1/$2@$0](om/.*@.*)s/.*/om/
+ RULE:[2:$1/$2@$0](scm/.*@.*)s/.*/scm/
+ RULE:[2:$1/$2@$0](dn/.*@.*)s/.*/dn/
+ RULE:[2:$1/$2@$0](solr/.*@.*)s/.*/solr/
+ RULE:[2:$1/$2@$0](atlas/.*@.*)s/.*/atlas/
+ RULE:[2:$1/$2@$0](kudu/.*@.*)s/.*/kudu/
+ RULE:[2:$1/$2@$0](nifi/.*@.*)s/.*/nifi/
RULE:[1:$1@$0](.*@.*)s/@.*//
DEFAULT
</value>
<description>
- Kerberos auth_to_local rules for mapping authenticated principals to service names.
- Uses Hadoop KerberosName syntax to convert full Kerberos principals to short usernames.
-
- IMPORTANT: These rules only apply to Kerberos/SPNEGO authentication. JWT and basic auth
- usernames are already in short form and pass through unchanged.
-
- Default rules provided:
- - RULE:[2:$1/$2@$0]([ndj]n/.*@.*|hdfs/.*@.*)s/.*/hdfs/ # nn,dn,jn,hdfs/* -> hdfs
- - RULE:[2:$1/$2@$0]([rn]m/.*@.*|yarn/.*@.*)s/.*/yarn/ # rm,nm,yarn/* -> yarn
- - RULE:[2:$1/$2@$0](jhs/.*@.*)s/.*/mapred/ # jhs/* -> mapred
- - RULE:[1:$1@$0](.*@.*)s/@.*// # user@REALM -> user
- - DEFAULT
+ Kerberos auth_to_local rules: map service/host@REALM to the short username checked
+ against ranger.audit.ingestor.service.<repo>.allowed.users (Policy Manager
+ policy.download.auth.users).
+
+ Order matters — service-specific RULE lines run before the user@REALM fallback.
+
+ Hadoop: nn/dn/jn/hdfs/* -> hdfs (dev_hdfs); rm/nm/yarn/* -> yarn (dev_yarn);
+ jhs/* -> mapred.
+ Plugins: hive/* -> hive (dev_hive); hbase/* -> hbase (dev_hbase);
+ kafka/* -> kafka (dev_kafka); knox/* -> knox (dev_knox);
+ rangerkms/* -> rangerkms (dev_kms); trino/* -> trino (dev_trino);
+ ozone/* -> ozone; om/* -> om; scm/* -> scm; dn/* -> dn (dev_ozone);
+ solr/* -> solr (dev_solr); atlas/* -> atlas (dev_atlas);
+ kudu/* -> kudu (dev_kudu); nifi/* -> nifi (dev_nifi).
+ Fallback: user@REALM -> user; then DEFAULT.
</description>
</property>
diff --git a/audit-server/scripts/README.md b/audit-server/scripts/README.md
index 6bd09a03b..917b0b02e 100644
--- a/audit-server/scripts/README.md
+++ b/audit-server/scripts/README.md
@@ -30,6 +30,8 @@ The codebase is organized into two main directories:
- `audit-ingestor/`
- `audit-dispatcher/` (contains a unified dispatcher application that dynamically loads specific dispatcher types)
+**Troubleshooting:** Plugin audits rejected with HTTP 403 from ingestor (KMS, Kafka, Ozone, etc.) — see [README-AUDIT-INGESTOR-SERVICE-ALLOWLIST.md](../README-AUDIT-INGESTOR-SERVICE-ALLOWLIST.md).
+
## Prerequisites
Before running these scripts, ensure you have:
diff --git a/dev-support/ranger-docker/scripts/admin/create-ranger-services.py b/dev-support/ranger-docker/scripts/admin/create-ranger-services.py
index 68d9b915e..6f85c2371 100644
--- a/dev-support/ranger-docker/scripts/admin/create-ranger-services.py
+++ b/dev-support/ranger-docker/scripts/admin/create-ranger-services.py
@@ -129,9 +129,9 @@ def service_not_exists(service):
'configs': {'username': 'hdfs', 'password': 'hdfs',
'ozone.om.http-address': 'http://om:9874',
'hadoop.security.authentication': 'simple',
- 'policy.download.auth.users': 'ozone',
- 'tag.download.auth.users': 'ozone',
- 'userstore.download.auth.users': 'ozone',
+ 'policy.download.auth.users': 'ozone,om,scm,dn',
+ 'tag.download.auth.users': 'ozone,om,scm,dn',
+ 'userstore.download.auth.users': 'ozone,om,scm,dn',
'ranger.plugin.ozone.policy.refresh.synchronous':'true'}})
solr = RangerService({'name': 'dev_solr', 'type': 'solr',
@@ -148,7 +148,32 @@ def service_not_exists(service):
'ranger.plugin.super.users': 'solr',
'ranger.plugin.solr.policy.refresh.synchronous':'true'}})
-services = [hdfs, yarn, hive, hbase, kafka, knox, kms, trino, ozone, solr]
+atlas = RangerService({'name': 'dev_atlas', 'type': 'atlas',
+ 'configs': {'username': 'atlas', 'password': 'atlas',
+ 'atlas.rest.address': 'http://atlas:21000',
+ 'policy.download.auth.users': 'atlas',
+ 'tag.download.auth.users': 'atlas',
+ 'userstore.download.auth.users': 'atlas',
+ 'ranger.plugin.atlas.policy.refresh.synchronous': 'true'}})
+
+kudu = RangerService({'name': 'dev_kudu', 'type': 'kudu',
+ 'configs': {'username': 'kudu', 'password': 'kudu',
+ 'policy.download.auth.users': 'kudu',
+ 'tag.download.auth.users': 'kudu',
+ 'userstore.download.auth.users': 'kudu',
+ 'ranger.plugin.kudu.policy.refresh.synchronous': 'true'}})
+
+nifi = RangerService({'name': 'dev_nifi', 'type': 'nifi',
+ 'configs': {'username': 'nifi', 'password': 'nifi',
+ 'nifi.url': 'http://ranger-nifi.rangernw:8080/nifi-api/resources',
+ 'nifi.authentication': 'NONE',
+ 'nifi.ssl.use.default.context': 'false',
+ 'policy.download.auth.users': 'nifi',
+ 'tag.download.auth.users': 'nifi',
+ 'userstore.download.auth.users': 'nifi',
+ 'ranger.plugin.nifi.policy.refresh.synchronous': 'true'}})
+
+services = [hdfs, yarn, hive, hbase, kafka, knox, kms, trino, ozone, solr, atlas, kudu, nifi]
for service in services:
try:
if service_not_exists(service):