Re: [PR] KAFKA-18646: Null records in fetch response breaks librdkafka [kafka]

["junrao (via GitHub)" <gi...@apache.org>]

junrao commented on code in PR #18726:
URL: https://github.com/apache/kafka/pull/18726#discussion_r1976167164


##########
clients/src/main/resources/common/message/FetchResponse.json:
##########
@@ -106,7 +106,7 @@
         ]},
         { "name": "PreferredReadReplica", "type": "int32", "versions": "11+", "default": "-1", "ignorable": false, "entityType": "brokerId",
           "about": "The preferred read replica for the consumer to use on its next fetch request."},
-        { "name": "Records", "type": "records", "versions": "0+", "nullableVersions": "0+", "about": "The record data."}

Review Comment:
   @chia7712 : My feeling is that we need a strong reason to change a schema without a KIP. Assuming that the above findings are correct, there is no strong reason. So, I'd recommend that we do the following in 4.0.
   
   1. Revert the schema change and make records nullable.
   2. Add a comment in the schema that records could be null only when errorCode is not 0.
   3. Ask librd to handle the potential null records correctly.
   
   Post 4.0, it would be useful to be more consistent on setting the values for records. We could (1) consistently set records to null for any error; or (2) make records non-nullable (ideally with a KIP). Either way, we probably want to handle the records in ShareFetchResponse in the same way.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org