[jira] [Comment Edited] (KAFKA-5390) First records in batch rejected but others accepted when rolling log

["Magnus Reftel (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16038879#comment-16038879 ] 

Magnus Reftel edited comment on KAFKA-5390 at 6/6/17 1:28 PM:
--------------------------------------------------------------

No, the server does not reject the batch - it rejects _some_ of the records in the batch, but not all. Further, the records of the batch that are accepted are not the first ones, so consumers would first see some records of earlier batches, then a gap where rejected records of the partially failing batch were, and then more records from the end of the batch.
The producer can detect that this happens from seeing that there are some failures followed by some successful sends, but it's too late for it to do anything about it.


was (Author: magnus.reftel):
No, the server does not reject the batch - it rejects /some/ of the records, but not all. Further, the records of the batch that are accepted are not the first ones, so consumers would first see some records of earlier batches, then a gap where rejected records of the partially failing batch were, and then more records from the end of the batch.
The producer can detect that this happens from seeing that there are some failures followed by some successful sends, but it's too late for it to do anything about it.

> First records in batch rejected but others accepted when rolling log
> --------------------------------------------------------------------
>
>                 Key: KAFKA-5390
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5390
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.10.2.1
>            Reporter: Magnus Reftel
>         Attachments: kafka-sequence-2.tar.gz, kafka-sequence.tar.gz
>
>
> When sending a sequence of records in a batch right when the broker needs to roll a new segment, it's possible for the first few records to fail, while other records in the batch are accepted. If records have dependencies on earlier records, e.g. in the case of a sequence of events in an event-sourced system, then a producer cannot use the batching functionality, since it then risks consumers receiving a record without first receiving the records it depends on.
> See attached testcase (kafka-sequence.tar.gz).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)