[jira] [Commented] (KAFKA-10465) Potential Bug/Doc update in Transactional Producer and Isolation Level

["Guozhang Wang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-10465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205236#comment-17205236 ] 

Guozhang Wang commented on KAFKA-10465:
---------------------------------------

Hi [~manmedia@gmail.com], hmm that is indeed a bit weird. I tried to reproduce what you've seen locally but did not succeed so far. I'd suggest you check the following things in order:

1) check on the broker's request log, and see that the *ListOffsetRequest* sent from endOffset() does have the *isolation_level* field set to read committed.

2) If 1) is true, then check on the broker side the LSO, which is the *firstUnstableOffsetMetadata* field in Log.scala, and see if it has been incremented to the returned offset indeed by the time the request is received.

3) If 3) is true, then you'd be able to find out the end-txn marker request that incremented the offset, and then you can check from txn coordinator on when it receives the EndOffsetRequest from producer that cause it to write the end-txn marker, from there you can find out when did the producer committed or aborted the txn, or if the txn is aborted proactively by the coordinator itself.

> Potential Bug/Doc update in Transactional Producer and Isolation Level
> ----------------------------------------------------------------------
>
>                 Key: KAFKA-10465
>                 URL: https://issues.apache.org/jira/browse/KAFKA-10465
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients
>    Affects Versions: 2.4.1
>            Reporter: M. Manna
>            Priority: Critical
>         Attachments: ConsumerTestApp.java, Consumer_endOffsets_return_84.jpg, ProducerTestApp.java, Producer_Committed_Successfully_to_82.jpg
>
>
> *Issue*
> Difference between LSO and High Watermark offsets when a consumer with "read_committed" aren't probably explained in the correct place.
> *Expected Behaviour*
> According to documentation, the offset returned should be the one committed last (and successfully). 
> *Observed (with steps)*
> 1. Start a local or test kafka cluster (2.4.1 or above)
> 2. Create a topic (I have used 3 replication-factor with 1 partition, but 1 and 1 is good)
> 3. Use the attached producer app file and set debug pointer to be able to pause on print
> 4. Use the attached consumer app file to start a consumer and debug through steps.
> It can be seen that the consumer is actually able to fetch an offset that's not committed by the producer yet. 
> Just trying to raise this ticket to confirm whether:
> 1) this is well-documented anywhere (which I have missed) - Please refer to this documentation as a resolution
> 2) This is a bug - please confirm and provide a timeline when this can be fixed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)