You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "kumarlokesh (via GitHub)" <gi...@apache.org> on 2025/05/23 19:51:45 UTC
[PR] KAFKA-19155: Update security guide with ACLs for stream RPCs [kafka]
kumarlokesh opened a new pull request, #19799:
URL: https://github.com/apache/kafka/pull/19799
Fixes https://issues.apache.org/jira/browse/KAFKA-19155
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Update stream RPCs security guide in developer documentation [kafka]
Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru closed pull request #19799: MINOR: Update stream RPCs security guide in developer documentation
URL: https://github.com/apache/kafka/pull/19799
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Update stream RPCs security guide in developer documentation [kafka]
Posted by "kumarlokesh (via GitHub)" <gi...@apache.org>.
kumarlokesh commented on PR #19799:
URL: https://github.com/apache/kafka/pull/19799#issuecomment-2943861487
@lucasbru thanks for the feedback! Updated documentation here https://github.com/apache/kafka/pull/19799/commits/3415d7ebe2042c9a990b10e0b774aa9bb8c2c9ab
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19155: Update security guide with ACLs for stream RPCs [kafka]
Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #19799:
URL: https://github.com/apache/kafka/pull/19799#discussion_r2123158533
##########
docs/streams/developer-guide/security.html:
##########
@@ -137,7 +138,49 @@
[...snip...]
Caused by: java.security.UnrecoverableKeyException: Password verification failed</code></pre>
<p>Monitor your Kafka Streams application log files for such error messages to spot any misconfigured applications quickly.</p>
-</div>
+ </div>
+ <div class="section" id="streams-rpc-acls">
+ <span id="streams-developer-guide-security-rpc-acls"></span><h2>Required ACLs for Streams RPCs<a class="headerlink" href="#streams-rpc-acls" title="Permalink to this headline"></a></h2>
+ <p>Kafka Streams uses several RPCs (Remote Procedure Calls) for its internal coordination and state management. The following ACLs are required for these RPCs to function properly:</p>
+
+ <h3>Streams Group Protocol RPCs</h3>
+ <p>For the Streams group protocol (introduced in <a class="reference external" href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol">KIP-1071</a>), the following ACLs are required:</p>
+ <pre><code class="language-bash"># Allow Streams clients to send heartbeats to the group coordinator
+kafka-acls --bootstrap-server <bootstrap_servers> \
Review Comment:
What's kafka_internal_streams_group_coordinator_topic here?
##########
docs/streams/developer-guide/security.html:
##########
@@ -137,7 +138,49 @@
[...snip...]
Caused by: java.security.UnrecoverableKeyException: Password verification failed</code></pre>
<p>Monitor your Kafka Streams application log files for such error messages to spot any misconfigured applications quickly.</p>
-</div>
+ </div>
+ <div class="section" id="streams-rpc-acls">
+ <span id="streams-developer-guide-security-rpc-acls"></span><h2>Required ACLs for Streams RPCs<a class="headerlink" href="#streams-rpc-acls" title="Permalink to this headline"></a></h2>
Review Comment:
Probably should mention streams rebalance protocol in the header.
##########
docs/streams/developer-guide/security.html:
##########
@@ -137,7 +138,49 @@
[...snip...]
Caused by: java.security.UnrecoverableKeyException: Password verification failed</code></pre>
<p>Monitor your Kafka Streams application log files for such error messages to spot any misconfigured applications quickly.</p>
-</div>
+ </div>
+ <div class="section" id="streams-rpc-acls">
+ <span id="streams-developer-guide-security-rpc-acls"></span><h2>Required ACLs for Streams RPCs<a class="headerlink" href="#streams-rpc-acls" title="Permalink to this headline"></a></h2>
+ <p>Kafka Streams uses several RPCs (Remote Procedure Calls) for its internal coordination and state management. The following ACLs are required for these RPCs to function properly:</p>
+
+ <h3>Streams Group Protocol RPCs</h3>
+ <p>For the Streams group protocol (introduced in <a class="reference external" href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol">KIP-1071</a>), the following ACLs are required:</p>
+ <pre><code class="language-bash"># Allow Streams clients to send heartbeats to the group coordinator
+kafka-acls --bootstrap-server <bootstrap_servers> \
+ --add --allow-principal User:<streams_app_principal> \
+ --operation WRITE \
+ --topic <kafka_internal_streams_group_coordinator_topic> \
+ --resource-pattern-type PREFIXED
+
+# Allow Streams clients to read from the group metadata topic
+kafka-acls --bootstrap-server <bootstrap_servers> \
+ --add --allow-principal User:<streams_app_principal> \
+ --operation READ \
+ --topic <kafka_internal_streams_group_metadata_topic> \
+ --resource-pattern-type PREFIXED
+
+# Allow Streams clients to describe groups
+kafka-acls --bootstrap-server <bootstrap_servers> \
+ --add --allow-principal User:<streams_app_principal> \
+ --operation DESCRIBE \
+ --group <application.id> \
+ --resource-pattern-type PREFIXED</code></pre>
+
+ <p>Replace <code><bootstrap_servers></code> with your Kafka brokers, <code><streams_app_principal></code> with the principal of your Streams application, and <code><application.id></code> with your Streams application ID.</p>
+
+ <h3>Interactive Queries RPCs</h3>
Review Comment:
I'm not sure this section is correct. Could you explain in more detail what you mean nby rest_api_principle? Would the rest api evne directly interact with kafka?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Update stream RPCs security guide in developer documentation [kafka]
Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on PR #19799:
URL: https://github.com/apache/kafka/pull/19799#issuecomment-3101787576
This was addressed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19155: Update security guide with ACLs for stream RPCs [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19799:
URL: https://github.com/apache/kafka/pull/19799#issuecomment-2924119324
A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the `triage` label
should be removed to prevent this automation from happening again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Update stream RPCs security guide in developer documentation [kafka]
Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #19799:
URL: https://github.com/apache/kafka/pull/19799#discussion_r2137256983
##########
docs/streams/developer-guide/security.html:
##########
@@ -137,7 +138,25 @@
[...snip...]
Caused by: java.security.UnrecoverableKeyException: Password verification failed</code></pre>
<p>Monitor your Kafka Streams application log files for such error messages to spot any misconfigured applications quickly.</p>
-</div>
+ </div>
+ <div class="section" id="streams-rpc-acls">
+ <span id="streams-developer-guide-security-rpc-acls"></span><h2>Security Considerations for Streams Rebalance Protocol<a class="headerlink" href="#streams-rpc-acls" title="Permalink to this headline"></a></h2>
+ <p>The Kafka Streams Rebalance Protocol (introduced in <a class="reference external" href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol">KIP-1071</a>) uses dedicated RPCs for group coordination. When running Streams applications in a secured Kafka environment, you will need to ensure proper authorization.</p>
+
+ <p>For detailed information about the required ACLs for the Streams Rebalance Protocol, refer to the <a class="reference internal" href="../../security.html#security_authz"><span class="std std-ref">Authorization and ACLs</span></a> section in the main Kafka Security documentation. This documentation lists all the operation permissions needed for various Kafka RPCs, including the Streams Group Heartbeat and Streams Group Describe operations.</p>
+
+ <h3>Interactive Queries Security</h3>
+ <p>If you are using <a class="reference internal" href="interactive-queries.html"><span class="doc">Interactive Queries</span></a> with a custom RPC layer (such as a REST API), you need to ensure that your application has the necessary permissions to access state stores and their changelog topics.</p>
+
+ <p>When implementing a custom query service that accesses state stores remotely, ensure that:</p>
+ <ul class="simple">
+ <li>The Kafka Streams application has permission to read from and write to the changelog topics that back the state stores</li>
+ <li>If your RPC service uses a different security principal (user/identity) than your Streams application, you must explicitly grant that principal the necessary permissions to read from the changelog topics</li>
Review Comment:
Again, why would the RPC service access the changelog topics?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org