[jira] [Commented] (KAFKA-8729) Augment ProduceResponse error messaging for specific culprit records

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16897561#comment-16897561 ] 

ASF GitHub Bot commented on KAFKA-8729:
---------------------------------------

tuvtran commented on pull request #7142: KAFKA-8729, pt 1: Add 4 new metrics to keep track of various types of invalid record rejections
URL: https://github.com/apache/kafka/pull/7142
 
 
   Right now we only have very generic `FailedProduceRequestsPerSec` and `FailedFetchRequestsPerSec` metrics that mark whenever a record is failed on the broker side. To improve the debugging UX, I added 4 new metrics in `BrokerTopicStats` to log various scenarios when an `InvalidRecordException` is thrown when `LogValidator` fails to validate a record:
   -- `NoKeyCompactedTopicRecordsPerSec`: counter of failures by compacted records with no key
   -- `InvalidMagicNumberRecordsPerSec`: counter of failures by records with invalid magic number
   -- `InvalidMessageCrcRecordsPerSec`: counter of failures by records with crc corruption
   -- `NonIncreasingOffsetRecordsPerSec`: counter of failures by records with invalid offset
   
   *More detailed description of your change,
   if necessary. The PR title and PR message become
   the squashed commit message, so use a separate
   comment to ping reviewers.*
   
   *Summary of testing strategy (including rationale)
   for the feature or bug fix. Unit and/or integration
   tests are expected for any behaviour change and
   system tests should be considered for larger changes.*
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Augment ProduceResponse error messaging for specific culprit records
> --------------------------------------------------------------------
>
>                 Key: KAFKA-8729
>                 URL: https://issues.apache.org/jira/browse/KAFKA-8729
>             Project: Kafka
>          Issue Type: Improvement
>          Components: clients, core
>            Reporter: Guozhang Wang
>            Assignee: Tu Tran
>            Priority: Major
>
> 1. We should replace the misleading CORRUPT_RECORD error code with a new INVALID_RECORD.
> 2. We should augment the ProduceResponse with customizable error message and indicators of culprit records.
> 3. We should change the client-side handling logic of non-retriable INVALID_RECORD to re-batch the records.
> Details see: https://cwiki.apache.org/confluence/display/KAFKA/KIP-467%3A+Augment+ProduceResponse+error+messaging+for+specific+culprit+records



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)