You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Eugen Feller (JIRA)" <ji...@apache.org> on 2018/09/12 19:16:00 UTC

[jira] [Updated] (KAFKA-7405) Support for graceful handling of corrupted records in Kafka consumer

     [ https://issues.apache.org/jira/browse/KAFKA-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eugen Feller updated KAFKA-7405:
--------------------------------
    Description: 
We have run into issues several times where corrupted records cause the Kafka consumer to throw an error code 2 exception (CRC checksum failure) in the fetch layer. Specifically, when using Kafka streams we run into KAFKA-6977 that throws an IllegalStateException and crashes the service. It would be great if the Kafka consumer could be extended with a setting similar to [KIP-161|https://cwiki.apache.org/confluence/display/KAFKA/KIP-161%3A+streams+deserialization+exception+handlers], that would allow one to gracefully ignore corrupted records.

 

  was:
We have run into issues several times where a corrupted records cause the Kafka consumer to throw an error code 2 exception in the fetch layer that can not be handled gracefully. Specifically, when using Kafka streams we run into KAFKA-6977 that throws an IllegalStateException. It would be great if the Kafka consumer could be extended with a setting similar to [KIP-161|https://cwiki.apache.org/confluence/display/KAFKA/KIP-161%3A+streams+deserialization+exception+handlers], that would allow one to cleanly ignore corrupted records.

 


> Support for graceful handling of corrupted records in Kafka consumer
> --------------------------------------------------------------------
>
>                 Key: KAFKA-7405
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7405
>             Project: Kafka
>          Issue Type: Improvement
>          Components: consumer, streams
>    Affects Versions: 0.10.2.2, 0.11.0.3, 1.1.1
>            Reporter: Eugen Feller
>            Priority: Major
>
> We have run into issues several times where corrupted records cause the Kafka consumer to throw an error code 2 exception (CRC checksum failure) in the fetch layer. Specifically, when using Kafka streams we run into KAFKA-6977 that throws an IllegalStateException and crashes the service. It would be great if the Kafka consumer could be extended with a setting similar to [KIP-161|https://cwiki.apache.org/confluence/display/KAFKA/KIP-161%3A+streams+deserialization+exception+handlers], that would allow one to gracefully ignore corrupted records.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)