You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ranger.apache.org by ra...@apache.org on 2026/06/15 06:13:16 UTC

(ranger) branch master updated: RANGER-5645: Add audit-ingestor service-user allowlist for Docker plugins (#1017)

This is an automated email from the ASF dual-hosted git repository.

ramackri pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ranger.git


The following commit(s) were added to refs/heads/master by this push:
     new 9ab006932 RANGER-5645: Add audit-ingestor service-user allowlist for Docker plugins (#1017)
9ab006932 is described below

commit 9ab0069321b9630b82f333e76c4a47e8b726298c
Author: Ramachandran Krishnan <ra...@gmail.com>
AuthorDate: Mon Jun 15 11:43:10 2026 +0530

    RANGER-5645: Add audit-ingestor service-user allowlist for Docker plugins (#1017)
    
    * RANGER-5645: Add audit-ingestor service-user allowlist for Docker plugins
    
    Ship per-repo allowed.users and auth_to_local rules so plugins using the
    audit-server destination are authorized after Kerberos SPNEGO (fixes HTTP 403).
    Align create-ranger-services.py with policy.download.auth.users for Ozone,
    Atlas, Kudu, and NiFi. Add troubleshooting README for ingestor 403 errors.
    
    Co-authored-by: Cursor <cu...@cursor.com>
    
    * RANGER-5645: Fix auth_to_local description; drop README files from PR
    
    Consolidate auth_to_local property description (JWT note + plugin rules).
    Revert audit-server/scripts/README.md and remove troubleshooting README.
    
    Co-authored-by: Cursor <cu...@cursor.com>
    
    * RANGER-5645: List auth_to_local rules in Default rules provided format
    
    Match original site XML description style with one bullet per RULE line.
    
    Co-authored-by: Cursor <cu...@cursor.com>
    
    * RANGER-5645: Limit scope to Docker services listed in Jira
    
    Remove dev_atlas, dev_kudu, and dev_nifi from ingestor allowlist,
    auth_to_local rules, and create-ranger-services.py (not in Docker stack).
    
    Co-authored-by: Cursor <cu...@cursor.com>
    
    * RANGER-5645: Add dev_elasticsearch for Docker OpenSearch stack
    
    Create Policy Manager repo for the elasticsearch service type pointing at
    ranger-opensearch.rangernw:9200 with opensearch download auth users, and
    add matching ingestor allowlist plus auth_to_local rule.
    
    Co-authored-by: Cursor <cu...@cursor.com>
    
    * RANGER-5645: Add dev_tag service and fix auth_to_local description
    
    Create dev_tag in create-ranger-services.py (matches Policy Manager).
    Add dev_tag ingestor allowlist (rangertagsync) and auth_to_local rule.
    Use CDATA so description shows <repo> instead of XML entities.
    
    Co-authored-by: Cursor <cu...@cursor.com>
    
    * RANGER-5645: Add atlas/kudu/nifi; remove OpenSearch elasticsearch entries
    
    Match Policy Manager repos (dev_atlas, dev_kudu, dev_nifi). Drop
    dev_elasticsearch allowlist, auth_to_local rules, and create-service entry.
    
    Co-authored-by: Cursor <cu...@cursor.com>
    
    * RANGER-5645: Address PR review feedback on allowlist and Docker services
    
    Remove redundant auth_to_local plugin rules covered by DEFAULT, restrict Ozone to om-only, and drop tag/atlas/kudu/nifi from create-ranger-services.py per Docker stack scope.
    
    Co-authored-by: Cursor <cu...@cursor.com>
    
    * RANGER-5645: Fix date-bound tag policy engine tests after 2026-06-15
    
    RESTRICTED-FINAL deny-exception uses isAccessedBefore(activation_date);
    fixture dates were 2026/06/15 so TestPolicyEngine_hiveForTag_filebased
    failed on/after that day (unrelated to ingestor allowlist changes).
    Use 2099/12/31 in tag test fixtures so CI stays stable.
    
    Co-authored-by: Cursor <cu...@cursor.com>
    
    * RANGER-5645: Run TestPolicyEngine sub-tests sequentially for CI stability
    
    parallelStream() on a shared RangerPolicyEngine caused intermittent
    failures (e.g. hdfs_resourcespec {USER} path policy) under CI load.
    
    Co-authored-by: Cursor <cu...@cursor.com>
    
    * RANGER-5645: Address review feedback on allowlist scope and test changes
    
    - Remove dev_tag, dev_atlas, dev_kudu, dev_nifi allowlist entries (not
      applicable to Docker audit-ingestor scope per review)
    - Revert agents-common test fixture and TestPolicyEngine changes; tag
      date updates belong in separate RANGER-5647 PR
    - Restore parallelStream() in TestPolicyEngine per review
    
    Co-authored-by: Cursor <cu...@cursor.com>
    
    ---------
    
    Co-authored-by: ramk <ra...@cloudera.com>
    Co-authored-by: Cursor <cu...@cursor.com>
---
 .../resources/conf/ranger-audit-ingestor-site.xml  | 60 +++++++++++++++++++---
 .../scripts/admin/create-ranger-services.py        |  6 +--
 2 files changed, 55 insertions(+), 11 deletions(-)

diff --git a/audit-server/audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml b/audit-server/audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml
index 976c2a0cb..1688b4dc4 100644
--- a/audit-server/audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml
+++ b/audit-server/audit-ingestor/src/main/resources/conf/ranger-audit-ingestor-site.xml
@@ -185,19 +185,61 @@
     <property>
         <name>ranger.audit.ingestor.service.dev_hdfs.allowed.users</name>
         <value>hdfs</value>
-        <description>Comma-separated list of allowed users that can send audits for dev_hdfs service</description>
+        <description>Allowed users for dev_hdfs (Policy Manager service name; from policy.download.auth.users)</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_yarn.allowed.users</name>
+        <value>yarn</value>
+        <description>Allowed users for dev_yarn (YARN RM/NM)</description>
     </property>
 
     <property>
         <name>ranger.audit.ingestor.service.dev_hive.allowed.users</name>
         <value>hive</value>
-        <description>Comma-separated list of allowed users that can send audits for dev_hive service.</description>
+        <description>Allowed users for dev_hive (HiveServer2 + Metastore)</description>
     </property>
 
     <property>
         <name>ranger.audit.ingestor.service.dev_hbase.allowed.users</name>
         <value>hbase</value>
-        <description>Comma-separated list of allowed users that can send audits for dev_hbase service.</description>
+        <description>Allowed users for dev_hbase (HBase Master + RegionServer)</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_kafka.allowed.users</name>
+        <value>kafka</value>
+        <description>Allowed users for dev_kafka</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_knox.allowed.users</name>
+        <value>knox</value>
+        <description>Allowed users for dev_knox</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_kms.allowed.users</name>
+        <value>rangerkms</value>
+        <description>Allowed users for dev_kms</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_trino.allowed.users</name>
+        <value>trino</value>
+        <description>Allowed users for dev_trino</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_ozone.allowed.users</name>
+        <value>om</value>
+        <description>Allowed users for dev_ozone (Ozone Manager; Ranger plugin runs on OM only)</description>
+    </property>
+
+    <property>
+        <name>ranger.audit.ingestor.service.dev_solr.allowed.users</name>
+        <value>solr</value>
+        <description>Allowed users for dev_solr (Solr plugin)</description>
     </property>
 
     <!-- AUTH_TO_LOCAL RULES FOR KERBEROS PRINCIPAL MAPPING -->
@@ -210,20 +252,22 @@
                RULE:[1:$1@$0](.*@.*)s/@.*//
                DEFAULT
         </value>
-        <description>
+        <description><![CDATA[
             Kerberos auth_to_local rules for mapping authenticated principals to service names.
-            Uses Hadoop KerberosName syntax to convert full Kerberos principals to short usernames.
+            Uses Hadoop KerberosName syntax to convert full Kerberos principals to short usernames
+            checked against ranger.audit.ingestor.service.<repo>.allowed.users (Policy Manager
+            policy.download.auth.users).
 
             IMPORTANT: These rules only apply to Kerberos/SPNEGO authentication. JWT and basic auth
             usernames are already in short form and pass through unchanged.
 
             Default rules provided:
-            - RULE:[2:$1/$2@$0]([ndj]n/.*@.*|hdfs/.*@.*)s/.*/hdfs/     # nn,dn,jn,hdfs/* -> hdfs
-            - RULE:[2:$1/$2@$0]([rn]m/.*@.*|yarn/.*@.*)s/.*/yarn/      # rm,nm,yarn/* -> yarn
+            - RULE:[2:$1/$2@$0]([ndj]n/.*@.*|hdfs/.*@.*)s/.*/hdfs/     # nn,dn,jn,hdfs/* -> hdfs (dev_hdfs)
+            - RULE:[2:$1/$2@$0]([rn]m/.*@.*|yarn/.*@.*)s/.*/yarn/      # rm,nm,yarn/* -> yarn (dev_yarn)
             - RULE:[2:$1/$2@$0](jhs/.*@.*)s/.*/mapred/                 # jhs/* -> mapred
             - RULE:[1:$1@$0](.*@.*)s/@.*//                             # user@REALM -> user
             - DEFAULT
-        </description>
+        ]]></description>
     </property>
 
 
diff --git a/dev-support/ranger-docker/scripts/admin/create-ranger-services.py b/dev-support/ranger-docker/scripts/admin/create-ranger-services.py
index 68d9b915e..d073faa5f 100644
--- a/dev-support/ranger-docker/scripts/admin/create-ranger-services.py
+++ b/dev-support/ranger-docker/scripts/admin/create-ranger-services.py
@@ -129,9 +129,9 @@ def service_not_exists(service):
                        'configs': {'username': 'hdfs', 'password': 'hdfs',
                                    'ozone.om.http-address': 'http://om:9874',
                                    'hadoop.security.authentication': 'simple',
-                                   'policy.download.auth.users': 'ozone',
-                                   'tag.download.auth.users': 'ozone',
-                                   'userstore.download.auth.users': 'ozone',
+                                   'policy.download.auth.users': 'om',
+                                   'tag.download.auth.users': 'om',
+                                   'userstore.download.auth.users': 'om',
                                    'ranger.plugin.ozone.policy.refresh.synchronous':'true'}})
 
 solr = RangerService({'name': 'dev_solr', 'type': 'solr',