You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Otavio Rodolfo Piske (Jira)" <ji...@apache.org> on 2021/09/02 09:52:00 UTC
[jira] [Updated] (CAMEL-16914) camel-kafka: possible corruption of
idempotency messages when using KafkaIdempotentRepository
[ https://issues.apache.org/jira/browse/CAMEL-16914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Otavio Rodolfo Piske updated CAMEL-16914:
-----------------------------------------
Attachment: MessageLossReproducer.java
> camel-kafka: possible corruption of idempotency messages when using KafkaIdempotentRepository
> ----------------------------------------------------------------------------------------------
>
> Key: CAMEL-16914
> URL: https://issues.apache.org/jira/browse/CAMEL-16914
> Project: Camel
> Issue Type: Task
> Components: camel-kafka
> Affects Versions: 3.11.1
> Reporter: Otavio Rodolfo Piske
> Priority: Major
> Attachments: FakeDataProducer.java, MessageLossReproducer.java
>
>
> I noticed that when using the KafkaIdempotentRepository, sometimes it seems that it is losing or corrupting the ID information, causing loss of messages.
>
> When reading the topic used for publishing the idempotency data with kafkacat I get:
>
> {code:java}
> ERROR: Failed to format message in fake-data-inserts [1] at offset 39: expected key end-of-input, but 2 bytes remaining : terminating{code}
>
> As a result, this causes the loss of messages because the consumers cannot go further than the corrupted record.
>
> I found this while using Camel K and Strimzi (on a setup with 3 Kafkas using 1 topic with 3 partitions).
>
> I couldn't test w/ 3.12 yet, but I suspect it would also have the same problem.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)