[jira] [Updated] (RANGER-5633) Audit-server Kafka tuning (producer, consumer, topic)

["Ramachandran Krishnan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/RANGER-5633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ramachandran Krishnan updated RANGER-5633:
------------------------------------------
    Attachment: Kafka consumers_ Solr + HDFS audit dispatchers.pdf
                Kafka producer performance (audit-ingestor).pdf
                Ranger Audit Server and Kafka_ KRaft vs ZooKeeper (3.9.1 Recommendation).pdf

> Audit-server Kafka tuning (producer, consumer, topic)
> -----------------------------------------------------
>
>                 Key: RANGER-5633
>                 URL: https://issues.apache.org/jira/browse/RANGER-5633
>             Project: Ranger
>          Issue Type: Task
>          Components: audit, Ranger
>            Reporter: Ramachandran Krishnan
>            Assignee: Ramachandran Krishnan
>            Priority: Major
>             Fix For: 3.0.0
>
>         Attachments: Kafka consumers_ Solr + HDFS audit dispatchers.pdf, Kafka producer performance (audit-ingestor).pdf, Ranger Audit Server and Kafka_ KRaft vs ZooKeeper (3.9.1 Recommendation).pdf
>
>
> Kafka producer and consumer configuration gaps in the audit-server stack:
>  * *Ingestor producer* — wire {{{}batch.size{}}}, {{{}linger.ms{}}}, {{{}buffer.memory{}}}, {{{}compression.type{}}}, timeouts, and related {{ProducerConfig}} keys from site XML; {{flush()}} on shutdown; configurable REST batch send timeout.
>  * *Dispatchers* — default to {{{}CooperativeStickyAssignor{}}}; document and ship consumer rebalance/poll timeout properties in Solr and HDFS site XML.
>  * *Topic creation* — optionally apply {{{}retention.ms{}}}, {{{}compression.type{}}}, and {{min.insync.replicas}} when Ranger creates the {{ranger_audits}} topic.
> All new properties are {*}backward compatible{*}: code defaults apply when site XML omits them.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)