You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by "Ramachandran Krishnan (Jira)" <ji...@apache.org> on 2026/08/31 17:32:00 UTC

[jira] [Updated] (RANGER-5719) Shared partition-plan library and SPIFFE header auth utilities

     [ https://issues.apache.org/jira/browse/RANGER-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ramachandran Krishnan updated RANGER-5719:
------------------------------------------
    Description: 
Ranger audit events are produced to a shared Kafka topic ({{{}ranger_audits{}}}). In dynamic routing mode, Admin owns a canonical partition plan: which plugin/service maps to which topic partitions, how buffer partitions are used for new services, and which users may POST audits per service repo. That plan must be represented, validated, and updated consistently across Admin and the audit ingestor.
h3. In scope

Partition plan model
 * JSON representation of the audit topic name, plan version, per-plugin partition assignments, buffer pool, and per-service allow-list metadata.

Plan lifecycle helpers
 * Allocation and validation when services are onboarded, offboarded, or reconfigured.
 * Append-only partition growth, buffer promotion, and version increments.

Routing utilities
 * Mapping from Admin-assigned logical partition IDs to Kafka producer partition indices, including safe behavior when live topic metadata differs from the plan.

Allow-list normalization
 * Parsing and normalization of per-repo allow-list values stored in the plan.
 * Allow-list usernames are sourced from {{service.admin.users}} (not {{{}policy.download.auth.users{}}}).

Admin configuration helpers
 * Reading partition-count defaults and per-plugin overrides used when building or expanding a plan.

  was:
h2. Summary

Introduces shared libraries in {{agents-common}} and {{common-utils}} that Admin, the Audit Ingestor, and plugins will use in follow-up PRs. This change adds building blocks only; no end-to-end audit routing or SPIFFE enforcement is active until later PRs land.
h3. Audit partition plan library

Adds a reusable partition-plan module for Admin-managed dynamic Kafka routing:
 * Plan model — JSON representation of the audit topic, plan version, per-plugin partition assignments, buffer pool, and per-service allow-list derived from {{{}policy.download.auth.users{}}}.
 * Plan lifecycle helpers — allocation and validation when services are onboarded or offboarded, including buffer promotion and version bumps.
 * Routing utilities — mapping from Admin-assigned partition IDs to Kafka producer partition indices, including safe fallbacks when topic metadata lags.
 * Plugin and service resolution — mapping Ranger service types to plugin IDs and parsing service config used for ingestor authorization.

This keeps partition-plan logic in one place so Admin, ingestor, and tests share the same contract.
h3. SPIFFE header auth utilities

Adds shared outbound auth helpers following the RANGER-5700 trusted-header pattern:
 * Header auth config — reads enable flag and SPIFFE header name from plugin/site configuration.
 * SPIFFE identity resolution — resolves workload identity from explicit config, identity file, or standard workload identity path.

These utilities support plugin → ingestor and ingestor → Admin SPIFFE auth in later PRs.
h3. What this PR does not include
 * Database schema or seed data
 * Admin REST APIs or plan persistence
 * Ingestor poller, dynamic partitioner, or inbound SPIFFE filter
 * Plugin install templates or audit destination changes
 * Docker or E2E smoke tests

Depends on: nothing
Blocks: DB patch, Admin, Ingestor, and Plugin PRs in the RANGER-5655 series


> Shared partition-plan library and SPIFFE header auth utilities
> --------------------------------------------------------------
>
>                 Key: RANGER-5719
>                 URL: https://issues.apache.org/jira/browse/RANGER-5719
>             Project: Ranger
>          Issue Type: Sub-task
>          Components: Ranger
>            Reporter: Ramachandran Krishnan
>            Assignee: Ramachandran Krishnan
>            Priority: Major
>             Fix For: 3.0.0
>
>          Time Spent: 50m
>  Remaining Estimate: 0h
>
> Ranger audit events are produced to a shared Kafka topic ({{{}ranger_audits{}}}). In dynamic routing mode, Admin owns a canonical partition plan: which plugin/service maps to which topic partitions, how buffer partitions are used for new services, and which users may POST audits per service repo. That plan must be represented, validated, and updated consistently across Admin and the audit ingestor.
> h3. In scope
> Partition plan model
>  * JSON representation of the audit topic name, plan version, per-plugin partition assignments, buffer pool, and per-service allow-list metadata.
> Plan lifecycle helpers
>  * Allocation and validation when services are onboarded, offboarded, or reconfigured.
>  * Append-only partition growth, buffer promotion, and version increments.
> Routing utilities
>  * Mapping from Admin-assigned logical partition IDs to Kafka producer partition indices, including safe behavior when live topic metadata differs from the plan.
> Allow-list normalization
>  * Parsing and normalization of per-repo allow-list values stored in the plan.
>  * Allow-list usernames are sourced from {{service.admin.users}} (not {{{}policy.download.auth.users{}}}).
> Admin configuration helpers
>  * Reading partition-count defaults and per-plugin overrides used when building or expanding a plan.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)