You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "robobario (via GitHub)" <gi...@apache.org> on 2024/11/27 21:41:28 UTC
[PR] MINOR: Add protocol documentation about ApiVersions Fallback behaviour [kafka]
robobario opened a new pull request, #17967:
URL: https://github.com/apache/kafka/pull/17967
Documents current ApiVersions fallback mechanism
The broker behaviour when it receives an ApiVersionsRequest ahead of it's supported versions is useful to know as an implementor/manipulator of the protocol.
The behaviour contradicts a point documented in the protocol guide. In the fallback case the protocol version of the response is different from the protocol version of the request.
Also spotted a couple of dead links in the table of contents
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Add protocol documentation about ApiVersions Fallback behaviour [kafka]
Posted by "robobario (via GitHub)" <gi...@apache.org>.
robobario commented on code in PR #17967:
URL: https://github.com/apache/kafka/pull/17967#discussion_r1861278947
##########
docs/protocol.html:
##########
@@ -143,7 +138,13 @@ <h5 class="anchor-heading"><a id="api_versions" class="anchor-link"></a><a href=
Kafka protocol requests may take place on an SSL listener before the SSL handshake is finished). If this is considered to
leak information about the broker version a workaround is to use SSL with client authentication which is performed at an
earlier stage of the connection where the <code>ApiVersionRequest</code> is not available. Also, note that broker versions older
- than 0.10.0.0 do not support this API and will either ignore the request or close connection in response to the request.</li>
+ than 0.10.0.0 do not support this API and will either ignore the request or close connection in response to the request. Also
Review Comment:
maybe better to extract it into it's own paragraph or section about fallback behaviour and link from here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Add protocol documentation about ApiVersions Fallback behaviour [kafka]
Posted by "robobario (via GitHub)" <gi...@apache.org>.
robobario commented on code in PR #17967:
URL: https://github.com/apache/kafka/pull/17967#discussion_r1870553518
##########
docs/protocol.html:
##########
@@ -143,7 +138,13 @@ <h5 class="anchor-heading"><a id="api_versions" class="anchor-link"></a><a href=
Kafka protocol requests may take place on an SSL listener before the SSL handshake is finished). If this is considered to
leak information about the broker version a workaround is to use SSL with client authentication which is performed at an
earlier stage of the connection where the <code>ApiVersionRequest</code> is not available. Also, note that broker versions older
- than 0.10.0.0 do not support this API and will either ignore the request or close connection in response to the request.</li>
+ than 0.10.0.0 do not support this API and will either ignore the request or close connection in response to the request. Also
+ note that if the client <code>ApiVersionsRequest</code> version is unsupported by the broker (client is ahead), and the broker
+ version is 2.5.0 or greater, then the broker will respond with a version 0 <a href="#The_Messages_ApiVersions">ApiVersionsResponse</a>
Review Comment:
I believe the comment is correct. I found 2.5.0 via the [2.5.0 release notes](https://archive.apache.org/dist/kafka/2.5.0/RELEASE_NOTES.html). This is because the JIRA ticket was partially implemented in 2.4.0 and finished in 2.5.0:
```
robeyoun@robeyoun-thinkpadp16vgen1:~/development/upstream/kafka$ git log 2.5.0 --grep KIP-511 --oneline
ae38315895 KAFKA-8855; Collect and Expose Client's Name and Version in the Brokers (KIP-511 Part 2) (#7749)
f98013cc50 Part 1 of KIP-511: Collect and Expose Client's Name and Version in the Brokers #7381
robeyoun@robeyoun-thinkpadp16vgen1:~/development/upstream/kafka$ git log 2.4.0 --grep KIP-511 --oneline
f98013cc50 Part 1 of KIP-511: Collect and Expose Client's Name and Version in the Brokers #7381
```
the relevant change to the ApiVersions response arrived in 2.4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Add protocol documentation about ApiVersions Fallback behaviour [kafka]
Posted by "robobario (via GitHub)" <gi...@apache.org>.
robobario commented on code in PR #17967:
URL: https://github.com/apache/kafka/pull/17967#discussion_r1870555818
##########
docs/protocol.html:
##########
@@ -143,7 +138,13 @@ <h5 class="anchor-heading"><a id="api_versions" class="anchor-link"></a><a href=
Kafka protocol requests may take place on an SSL listener before the SSL handshake is finished). If this is considered to
leak information about the broker version a workaround is to use SSL with client authentication which is performed at an
earlier stage of the connection where the <code>ApiVersionRequest</code> is not available. Also, note that broker versions older
- than 0.10.0.0 do not support this API and will either ignore the request or close connection in response to the request.</li>
+ than 0.10.0.0 do not support this API and will either ignore the request or close connection in response to the request. Also
+ note that if the client <code>ApiVersionsRequest</code> version is unsupported by the broker (client is ahead), and the broker
+ version is 2.5.0 or greater, then the broker will respond with a version 0 <a href="#The_Messages_ApiVersions">ApiVersionsResponse</a>
Review Comment:
pushed up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Add protocol documentation about ApiVersions Fallback behaviour [kafka]
Posted by "showuon (via GitHub)" <gi...@apache.org>.
showuon commented on code in PR #17967:
URL: https://github.com/apache/kafka/pull/17967#discussion_r1867366368
##########
docs/protocol.html:
##########
@@ -143,7 +138,13 @@ <h5 class="anchor-heading"><a id="api_versions" class="anchor-link"></a><a href=
Kafka protocol requests may take place on an SSL listener before the SSL handshake is finished). If this is considered to
leak information about the broker version a workaround is to use SSL with client authentication which is performed at an
earlier stage of the connection where the <code>ApiVersionRequest</code> is not available. Also, note that broker versions older
- than 0.10.0.0 do not support this API and will either ignore the request or close connection in response to the request.</li>
+ than 0.10.0.0 do not support this API and will either ignore the request or close connection in response to the request. Also
+ note that if the client <code>ApiVersionsRequest</code> version is unsupported by the broker (client is ahead), and the broker
+ version is 2.5.0 or greater, then the broker will respond with a version 0 <a href="#The_Messages_ApiVersions">ApiVersionsResponse</a>
Review Comment:
I see it's 2.4 [here](https://github.com/apache/kafka/blob/f87c04854bc532247542f26af19198eb879c2ab6/clients/src/main/java/org/apache/kafka/common/requests/ApiVersionsRequest.java#L120). Which one is correct?
##########
docs/protocol.html:
##########
@@ -143,7 +138,13 @@ <h5 class="anchor-heading"><a id="api_versions" class="anchor-link"></a><a href=
Kafka protocol requests may take place on an SSL listener before the SSL handshake is finished). If this is considered to
leak information about the broker version a workaround is to use SSL with client authentication which is performed at an
earlier stage of the connection where the <code>ApiVersionRequest</code> is not available. Also, note that broker versions older
- than 0.10.0.0 do not support this API and will either ignore the request or close connection in response to the request.</li>
+ than 0.10.0.0 do not support this API and will either ignore the request or close connection in response to the request. Also
+ note that if the client <code>ApiVersionsRequest</code> version is unsupported by the broker (client is ahead), and the broker
+ version is 2.5.0 or greater, then the broker will respond with a version 0 <a href="#The_Messages_ApiVersions">ApiVersionsResponse</a>
Review Comment:
Where is the `The_Messages_ApiVersions`? I can't find it anywhere.
##########
docs/protocol.html:
##########
@@ -45,11 +45,6 @@ <h1>Kafka protocol guide</h1>
<li><a href="#protocol_recordbatch">Record Batch</a>
</ul>
</li>
- <li><a href="#protocol_evolution">Evolving the Protocol</a>
- <ul>
- <li><a href="#protocol_versioning">The Request Header</a>
- <li><a href="#protocol_versioning">Versioning</a>
Review Comment:
Nice catch!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Add protocol documentation about ApiVersions Fallback behaviour [kafka]
Posted by "robobario (via GitHub)" <gi...@apache.org>.
robobario commented on code in PR #17967:
URL: https://github.com/apache/kafka/pull/17967#discussion_r1870554713
##########
docs/protocol.html:
##########
@@ -143,7 +138,13 @@ <h5 class="anchor-heading"><a id="api_versions" class="anchor-link"></a><a href=
Kafka protocol requests may take place on an SSL listener before the SSL handshake is finished). If this is considered to
leak information about the broker version a workaround is to use SSL with client authentication which is performed at an
earlier stage of the connection where the <code>ApiVersionRequest</code> is not available. Also, note that broker versions older
- than 0.10.0.0 do not support this API and will either ignore the request or close connection in response to the request.</li>
+ than 0.10.0.0 do not support this API and will either ignore the request or close connection in response to the request. Also
+ note that if the client <code>ApiVersionsRequest</code> version is unsupported by the broker (client is ahead), and the broker
+ version is 2.5.0 or greater, then the broker will respond with a version 0 <a href="#The_Messages_ApiVersions">ApiVersionsResponse</a>
Review Comment:
it's generated, see the current docs for an example https://kafka.apache.org/protocol#The_Messages_ApiVersions
Locally I had to do something like `./gradlew siteDocsTar` and follow the instructions on [kafka-site ](https://github.com/apache/kafka-site?tab=readme-ov-file#how-to-preview-the-latest-documentation-changes-in-kafka-repository)
to get the whole page rendered
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Add protocol documentation about ApiVersions Fallback behaviour [kafka]
Posted by "robobario (via GitHub)" <gi...@apache.org>.
robobario commented on code in PR #17967:
URL: https://github.com/apache/kafka/pull/17967#discussion_r1870553518
##########
docs/protocol.html:
##########
@@ -143,7 +138,13 @@ <h5 class="anchor-heading"><a id="api_versions" class="anchor-link"></a><a href=
Kafka protocol requests may take place on an SSL listener before the SSL handshake is finished). If this is considered to
leak information about the broker version a workaround is to use SSL with client authentication which is performed at an
earlier stage of the connection where the <code>ApiVersionRequest</code> is not available. Also, note that broker versions older
- than 0.10.0.0 do not support this API and will either ignore the request or close connection in response to the request.</li>
+ than 0.10.0.0 do not support this API and will either ignore the request or close connection in response to the request. Also
+ note that if the client <code>ApiVersionsRequest</code> version is unsupported by the broker (client is ahead), and the broker
+ version is 2.5.0 or greater, then the broker will respond with a version 0 <a href="#The_Messages_ApiVersions">ApiVersionsResponse</a>
Review Comment:
I believe the comment is correct. I found 2.5.0 via the [2.5.0 release notes](https://archive.apache.org/dist/kafka/2.5.0/RELEASE_NOTES.html). This is because the JIRA ticket was partially implemented in 2.4.0 and finished in 2.5.0:
```
$ git log 2.5.0 --grep KIP-511 --oneline
ae38315895 KAFKA-8855; Collect and Expose Client's Name and Version in the Brokers (KIP-511 Part 2) (#7749)
f98013cc50 Part 1 of KIP-511: Collect and Expose Client's Name and Version in the Brokers #7381
$ git log 2.4.0 --grep KIP-511 --oneline
f98013cc50 Part 1 of KIP-511: Collect and Expose Client's Name and Version in the Brokers #7381
```
the relevant change to the ApiVersions response arrived in 2.4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Add protocol documentation about ApiVersions Fallback behaviour [kafka]
Posted by "showuon (via GitHub)" <gi...@apache.org>.
showuon merged PR #17967:
URL: https://github.com/apache/kafka/pull/17967
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org