You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "mumrah (via GitHub)" <gi...@apache.org> on 2023/02/03 21:32:00 UTC
[GitHub] [kafka] mumrah opened a new pull request, #13197: Minor: Decode the envelope requests for the request log
mumrah opened a new pull request, #13197:
URL: https://github.com/apache/kafka/pull/13197
When looking at the request log for anything forwarded to the controller, all we see is ENVELOPE and the base64 encoded bytes of the request body.
This patch decodes the envelope request so it can be fully logged in the request log.
Here is a sample output before
> [2023-02-03 16:29:55,747] DEBUG Completed request:{"isForwarded":false,"requestHeader":{"requestApiKey":58,"requestApiVersion":0,"correlationId":6,"clientId":"1","requestApiKeyName":"ENVELOPE"},"request":{"requestData":"ACwAAQAAAAMADWFkbWluY2xpZW50LTIAAgIFdGVzdAMOc2VnbWVudC5ieXRlcwAHMjA0ODAwAAtzZWdtZW50Lm1zAQAAAAAA","requestPrincipal":"AAAFVXNlcgpBTk9OWU1PVVMAAA==","clientHostAddress":"fwAAAQ=="},"response":{"responseData":null,"errorCode":41},"connection":"127.0.0.1:51244-127.0.0.1:51264-1","totalTimeMs":0.362,"requestQueueTimeMs":0.096,"localTimeMs":0.12,"remoteTimeMs":0.0,"throttleTimeMs":0,"responseQueueTimeMs":0.051,"sendTimeMs":0.095,"securityProtocol":"PLAINTEXT","principal":"User:ANONYMOUS","listener":"EXTERNAL","clientInformation":{"softwareName":"apache-kafka-java","softwareVersion":"unknown"}} (kafka.request.logger:275)
and after:
> [2023-02-03 16:25:17,606] DEBUG Completed request:{"isForwarded":false,"requestHeader":{"requestApiKey":58,"requestApiVersion":0,"correlationId":0,"clientId":"0","requestApiKeyName":"ENVELOPE"},"request":{"envelopeRequestHeader":{"requestApiKey":44,"requestApiVersion":1,"correlationId":3,"clientId":"adminclient-2","requestApiKeyName":"INCREMENTAL_ALTER_CONFIGS"},"envelopeRequest":{"resources":[{"resourceType":2,"resourceName":"test","configs":[{"name":"segment.bytes","configOperation":0,"value":"204800"},{"name":"segment.ms","configOperation":1,"value":null}]}],"validateOnly":false}},"response":{"responseData":null,"errorCode":41},"connection":"127.0.0.1:50795-127.0.0.1:50813-0","totalTimeMs":4.067,"requestQueueTimeMs":1.86,"localTimeMs":1.927,"remoteTimeMs":0.0,"throttleTimeMs":0,"responseQueueTimeMs":0.096,"sendTimeMs":0.183,"securityProtocol":"PLAINTEXT","principal":"User:ANONYMOUS","listener":"EXTERNAL","clientInformation":{"softwareName":"apache-kafka-java","softwareVersion
":"unknown"}} (kafka.request.logger:275)
This change slightly increases the logging overhead since it makes a copy of the request buffer and runs through the parsing logic. However, this seems like a reasonable tradeoff for better visibility.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jsancio commented on a diff in pull request #13197: Minor: Decode the envelope requests for the request log
Posted by "jsancio (via GitHub)" <gi...@apache.org>.
jsancio commented on code in PR #13197:
URL: https://github.com/apache/kafka/pull/13197#discussion_r1097782211
##########
core/src/main/scala/kafka/network/RequestConvertToJson.scala:
##########
@@ -60,7 +61,16 @@ object RequestConvertToJson {
case req: ElectLeadersRequest => ElectLeadersRequestDataJsonConverter.write(req.data, request.version)
case req: EndTxnRequest => EndTxnRequestDataJsonConverter.write(req.data, request.version)
case req: EndQuorumEpochRequest => EndQuorumEpochRequestDataJsonConverter.write(req.data, request.version)
- case req: EnvelopeRequest => EnvelopeRequestDataJsonConverter.write(req.data, request.version)
+ case req: EnvelopeRequest => {
+ val envelopeRequestData = req.data()
+ val envelopeData = envelopeRequestData.requestData().duplicate()
+ val envelopeHeader = EnvelopeUtils.parseForwardedRequestHeader(envelopeData)
+ val requestJson = RequestConvertToJson.request(AbstractRequest.parseRequest(envelopeHeader.apiKey(), envelopeHeader.apiVersion(), envelopeData).request)
Review Comment:
Hey @mumrah, does this parse the enveloped request twice? If so, is it possible to implement this feature without parsing the enveloped request twice?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] github-actions[bot] commented on pull request #13197: Minor: Decode the envelope requests for the request log
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13197:
URL: https://github.com/apache/kafka/pull/13197#issuecomment-1585813236
This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge conflicts, please update it with the latest from trunk (or appropriate release branch)
If this PR is no longer valid or desired, please feel free to close it. If no activity occurrs in the next 30 days, it will be automatically closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org