You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "KTKTK-HZ (via GitHub)" <gi...@apache.org> on 2025/07/02 03:23:33 UTC

[PR] KAFKA-19377:Update /streams/developer-guide/security.html for KIP-1071 [kafka]

KTKTK-HZ opened a new pull request, #20084:
URL: https://github.com/apache/kafka/pull/20084

   (no comment)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19377:Update /streams/developer-guide/security.html for KIP-1071 [kafka]

Posted by "KTKTK-HZ (via GitHub)" <gi...@apache.org>.
KTKTK-HZ commented on PR #20084:
URL: https://github.com/apache/kafka/pull/20084#issuecomment-3052788140

   @lucasbru, thank you for your comments. I've updated the document based on the latest KIP specifications. Additionally, I've included the rendering outcomes of the new document.
   Could you please review it and let me know if there are any further adjustments required?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19377:Update /streams/developer-guide/security.html for KIP-1071 [kafka]

Posted by "KTKTK-HZ (via GitHub)" <gi...@apache.org>.
KTKTK-HZ commented on PR #20084:
URL: https://github.com/apache/kafka/pull/20084#issuecomment-3030224800

   hi, @lucasbru ,Would you mind reviewing this pull request?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19377:Update /streams/developer-guide/security.html for KIP-1071 [kafka]

Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #20084:
URL: https://github.com/apache/kafka/pull/20084#discussion_r2197192671


##########
docs/streams/developer-guide/security.html:
##########
@@ -70,7 +70,65 @@
                 the ACL set so that the application has the permissions to create, read and write
                 <a class="reference internal" href="manage-topics.html#streams-developer-guide-topics-internal"><span class="std std-ref">internal topics</span></a>.</p>
 
-                <p>To avoid providing this permission to your application, you can create the required internal topics manually.
+            <div class="admonition">
+                <p>Starting with <a class="reference external" href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol">KIP-1071</a>(Currently, it's in the Early Access phase), additional ACLs are required for the new Streams rebalance protocol. You can enable this protocol by using the <code>group.protocol=streams</code> configuration:</p>
+            </div>
+
+            <table border="1" class="docutils">
+                <colgroup>
+                    <col width="25%">
+                    <col width="15%">
+                    <col width="20%">
+                    <col width="40%">
+                </colgroup>
+                <thead valign="bottom">
+                <tr class="row-odd"><th class="head">API PROTOCOL</th>
+                    <th class="head">OPERATION</th>
+                    <th class="head">Resource</th>
+                    <th class="head">Notes</th>
+                </tr>
+                </thead>
+                <tbody valign="top">
+                <tr class="row-even">
+                    <td>STREAMS_GROUP_HEARTBEAT</td>
+                    <td>Read</td>
+                    <td>Group</td>
+                    <td>Required for the application's streams groups</td>

Review Comment:
   ```suggestion
                       <td>Required for the application's streams group</td>
   ```



##########
docs/streams/developer-guide/security.html:
##########
@@ -70,7 +70,65 @@
                 the ACL set so that the application has the permissions to create, read and write
                 <a class="reference internal" href="manage-topics.html#streams-developer-guide-topics-internal"><span class="std std-ref">internal topics</span></a>.</p>
 
-                <p>To avoid providing this permission to your application, you can create the required internal topics manually.
+            <div class="admonition">
+                <p>Starting with <a class="reference external" href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol">KIP-1071</a>(Currently, it's in the Early Access phase), additional ACLs are required for the new Streams rebalance protocol. You can enable this protocol by using the <code>group.protocol=streams</code> configuration:</p>
+            </div>
+
+            <table border="1" class="docutils">
+                <colgroup>
+                    <col width="25%">
+                    <col width="15%">
+                    <col width="20%">
+                    <col width="40%">
+                </colgroup>
+                <thead valign="bottom">
+                <tr class="row-odd"><th class="head">API PROTOCOL</th>
+                    <th class="head">OPERATION</th>
+                    <th class="head">Resource</th>
+                    <th class="head">Notes</th>
+                </tr>
+                </thead>
+                <tbody valign="top">
+                <tr class="row-even">
+                    <td>STREAMS_GROUP_HEARTBEAT</td>
+                    <td>Read</td>
+                    <td>Group</td>
+                    <td>Required for the application's streams groups</td>
+                </tr>
+                <tr class="row-odd">
+                    <td>STREAMS_GROUP_HEARTBEAT</td>
+                    <td>Create</td>
+                    <td>Cluster <i>or</i> Topic</td>
+                    <td>
+                        Required only if auto-creating internal topics.<br>
+                        • <code>Create</code> on Cluster resource<br>
+                        • or <code>Create</code> on all topics in StateChangelogTopics and RepartitionSourceTopics<br>
+                        Not required if internal topics are pre-created
+                    </td>
+                </tr>
+                <tr class="row-even">
+                    <td>STREAMS_GROUP_HEARTBEAT</td>
+                    <td>Describe</td>
+                    <td>Topic</td>
+                    <td>Required for all topics included in the message</td>
+                </tr>
+                <tr class="row-odd">
+                    <td>STREAMS_GROUP_DESCRIBE</td>
+                    <td>Describe</td>
+                    <td>Group</td>
+                    <td>Required for the application's streams groups</td>

Review Comment:
   ```suggestion
                       <td>Required for the application's streams group</td>
   ```



##########
docs/streams/developer-guide/security.html:
##########
@@ -70,7 +70,65 @@
                 the ACL set so that the application has the permissions to create, read and write
                 <a class="reference internal" href="manage-topics.html#streams-developer-guide-topics-internal"><span class="std std-ref">internal topics</span></a>.</p>
 
-                <p>To avoid providing this permission to your application, you can create the required internal topics manually.
+            <div class="admonition">
+                <p>Starting with <a class="reference external" href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol">KIP-1071</a>(Currently, it's in the Early Access phase), additional ACLs are required for the new Streams rebalance protocol. You can enable this protocol by using the <code>group.protocol=streams</code> configuration:</p>
+            </div>
+
+            <table border="1" class="docutils">
+                <colgroup>
+                    <col width="25%">
+                    <col width="15%">
+                    <col width="20%">
+                    <col width="40%">
+                </colgroup>
+                <thead valign="bottom">
+                <tr class="row-odd"><th class="head">API PROTOCOL</th>
+                    <th class="head">OPERATION</th>
+                    <th class="head">Resource</th>
+                    <th class="head">Notes</th>
+                </tr>
+                </thead>
+                <tbody valign="top">
+                <tr class="row-even">
+                    <td>STREAMS_GROUP_HEARTBEAT</td>
+                    <td>Read</td>
+                    <td>Group</td>
+                    <td>Required for the application's streams groups</td>
+                </tr>
+                <tr class="row-odd">
+                    <td>STREAMS_GROUP_HEARTBEAT</td>
+                    <td>Create</td>
+                    <td>Cluster <i>or</i> Topic</td>
+                    <td>
+                        Required only if auto-creating internal topics.<br>
+                        • <code>Create</code> on Cluster resource<br>
+                        • or <code>Create</code> on all topics in StateChangelogTopics and RepartitionSourceTopics<br>
+                        Not required if internal topics are pre-created
+                    </td>
+                </tr>
+                <tr class="row-even">
+                    <td>STREAMS_GROUP_HEARTBEAT</td>
+                    <td>Describe</td>
+                    <td>Topic</td>
+                    <td>Required for all topics included in the message</td>

Review Comment:
   ```suggestion
                       <td>Required for all topics used in the application's topology, when first joining.</td>
   ```



##########
docs/streams/developer-guide/security.html:
##########
@@ -70,7 +70,65 @@
                 the ACL set so that the application has the permissions to create, read and write
                 <a class="reference internal" href="manage-topics.html#streams-developer-guide-topics-internal"><span class="std std-ref">internal topics</span></a>.</p>
 
-                <p>To avoid providing this permission to your application, you can create the required internal topics manually.
+            <div class="admonition">
+                <p>Starting with <a class="reference external" href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol">KIP-1071</a>(Currently, it's in the Early Access phase), additional ACLs are required for the new Streams rebalance protocol. You can enable this protocol by using the <code>group.protocol=streams</code> configuration:</p>

Review Comment:
   There is a space missing. Also, it's enough for this to be included in 4.2 documentation, at that point we are not in EA anymore.
   
   ```suggestion
                   <p>If the <a class="reference external" href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol">streams rebalance protocol</a> is enabled by setting <code>group.protocol=streams</code>, the following ACLs are required on the topic and group resources:</p>
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19377:Update /streams/developer-guide/security.html for KIP-1071 [kafka]

Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru merged PR #20084:
URL: https://github.com/apache/kafka/pull/20084


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19377:Update /streams/developer-guide/security.html for KIP-1071 [kafka]

Posted by "KTKTK-HZ (via GitHub)" <gi...@apache.org>.
KTKTK-HZ commented on PR #20084:
URL: https://github.com/apache/kafka/pull/20084#issuecomment-3077779986

   Hi, @lucasbru ,Are there any other changes needed for this PR?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19377:Update /streams/developer-guide/security.html for KIP-1071 [kafka]

Posted by "KTKTK-HZ (via GitHub)" <gi...@apache.org>.
KTKTK-HZ commented on PR #20084:
URL: https://github.com/apache/kafka/pull/20084#issuecomment-3057729468

   @lucasbru Thanks for your suggestions. The document has been revised, and the rendered results of the document have also been updated.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19377:Update /streams/developer-guide/security.html for KIP-1071 [kafka]

Posted by "KTKTK-HZ (via GitHub)" <gi...@apache.org>.
KTKTK-HZ commented on code in PR #20084:
URL: https://github.com/apache/kafka/pull/20084#discussion_r2195090700


##########
docs/streams/developer-guide/security.html:
##########
@@ -70,6 +70,58 @@
                 the ACL set so that the application has the permissions to create, read and write
                 <a class="reference internal" href="manage-topics.html#streams-developer-guide-topics-internal"><span class="std std-ref">internal topics</span></a>.</p>
 
+            <div class="admonition note">
+                <p class="first admonition-title">Note</p>
+                <p class="last">Starting with <a class="reference external" href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol">KIP-1071</a>, additional ACLs are required for the new Streams rebalance protocol:</p>
+            </div>
+
+            <table border="1" class="docutils">
+                <colgroup>
+                    <col width="25%">
+                    <col width="15%">
+                    <col width="20%">
+                    <col width="40%">
+                </colgroup>
+                <thead valign="bottom">
+                <tr class="row-odd"><th class="head">Operation (API)</th>
+                    <th class="head">Permission</th>
+                    <th class="head">Resource</th>
+                    <th class="head">Notes</th>
+                </tr>
+                </thead>
+                <tbody valign="top">
+                <tr class="row-even">
+                    <td>STREAMS_GROUP_HEARTBEAT</td>
+                    <td>Read</td>
+                    <td>Group</td>
+                    <td>Required for the application's consumer group (<code>${application.id}</code>)</td>
+                </tr>
+                <tr class="row-odd">
+                    <td>STREAMS_GROUP_HEARTBEAT</td>
+                    <td>DescribeConfigs</td>

Review Comment:
   Thanks for the reminder.The document has been revised according to the description in the new KIP.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19377:Update /streams/developer-guide/security.html for KIP-1071 [kafka]

Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on PR #20084:
URL: https://github.com/apache/kafka/pull/20084#issuecomment-3058639838

   
   <img width="960" height="683" alt="image" src="https://github.com/user-attachments/assets/79a4674e-fcb6-4d8e-9650-616978064bac" />
   
   Link to rendering, removed from PR descritpipn


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19377:Update /streams/developer-guide/security.html for KIP-1071 [kafka]

Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #20084:
URL: https://github.com/apache/kafka/pull/20084#discussion_r2190511770


##########
docs/streams/developer-guide/security.html:
##########
@@ -70,6 +70,58 @@
                 the ACL set so that the application has the permissions to create, read and write
                 <a class="reference internal" href="manage-topics.html#streams-developer-guide-topics-internal"><span class="std std-ref">internal topics</span></a>.</p>
 
+            <div class="admonition note">
+                <p class="first admonition-title">Note</p>
+                <p class="last">Starting with <a class="reference external" href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol">KIP-1071</a>, additional ACLs are required for the new Streams rebalance protocol:</p>
+            </div>
+
+            <table border="1" class="docutils">
+                <colgroup>
+                    <col width="25%">
+                    <col width="15%">
+                    <col width="20%">
+                    <col width="40%">
+                </colgroup>
+                <thead valign="bottom">
+                <tr class="row-odd"><th class="head">Operation (API)</th>
+                    <th class="head">Permission</th>
+                    <th class="head">Resource</th>
+                    <th class="head">Notes</th>
+                </tr>
+                </thead>
+                <tbody valign="top">
+                <tr class="row-even">
+                    <td>STREAMS_GROUP_HEARTBEAT</td>
+                    <td>Read</td>
+                    <td>Group</td>
+                    <td>Required for the application's consumer group (<code>${application.id}</code>)</td>
+                </tr>
+                <tr class="row-odd">
+                    <td>STREAMS_GROUP_HEARTBEAT</td>
+                    <td>DescribeConfigs</td>

Review Comment:
   DESCRIBE, not DESCRIBE_CONFIGS



##########
docs/streams/developer-guide/security.html:
##########
@@ -88,6 +140,23 @@
                 (see <a class="reference external" href="https://cwiki.apache.org/confluence/x/zlOHB">KIP-277</a>
                 and <a class="reference external" href="https://cwiki.apache.org/confluence/x/QpvLB">KIP-290</a> for details).
             </p>
+            <div class="admonition best-practice">
+                <p class="first admonition-title">Best Practice</p>
+                <p class="last">For secure Streams deployments:
+                <ul class="simple">
+                    <li><strong>Pre-create internal topics</strong> to avoid requiring <code>CREATE</code> permissions</li>
+                    <li>Grant <strong>minimal required permissions</strong>:
+                        <ul>
+                            <li><code>READ</code> on group <code>${application.id}</code></li>
+                            <li><code>DESCRIBE_CONFIGS</code> on topics</li>

Review Comment:
   The KIP wasn't quite up-to-date. We require DESCRIBE on all topics involved in the group topology



##########
docs/streams/developer-guide/security.html:
##########
@@ -70,6 +70,58 @@
                 the ACL set so that the application has the permissions to create, read and write
                 <a class="reference internal" href="manage-topics.html#streams-developer-guide-topics-internal"><span class="std std-ref">internal topics</span></a>.</p>
 
+            <div class="admonition note">
+                <p class="first admonition-title">Note</p>
+                <p class="last">Starting with <a class="reference external" href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol">KIP-1071</a>, additional ACLs are required for the new Streams rebalance protocol:</p>

Review Comment:
   We need to state that this only applies when `group.protocol=streams` is set in the configuration.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19377:Update /streams/developer-guide/security.html for KIP-1071 [kafka]

Posted by "KTKTK-HZ (via GitHub)" <gi...@apache.org>.
KTKTK-HZ commented on code in PR #20084:
URL: https://github.com/apache/kafka/pull/20084#discussion_r2195091326


##########
docs/streams/developer-guide/security.html:
##########
@@ -70,6 +70,58 @@
                 the ACL set so that the application has the permissions to create, read and write
                 <a class="reference internal" href="manage-topics.html#streams-developer-guide-topics-internal"><span class="std std-ref">internal topics</span></a>.</p>
 
+            <div class="admonition note">
+                <p class="first admonition-title">Note</p>
+                <p class="last">Starting with <a class="reference external" href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol">KIP-1071</a>, additional ACLs are required for the new Streams rebalance protocol:</p>

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org