[GitHub] [kafka] vamossagar12 commented on pull request #13646: KAFKA-14938: Fixing flaky test testConnectorBoundary

["vamossagar12 (via GitHub)" <gi...@apache.org>]

vamossagar12 commented on PR #13646:
URL: https://github.com/apache/kafka/pull/13646#issuecomment-1591623006

   Thanks @sudeshwasnik , for the assessment. Continuing the line of thought, if you check the `MonitorableSourceConnector#poll` method, it appears to me that we decrement the `recordsRemainingLatch` latch equal to the number of records in the batch irrespective of the fact that whether the records are going to be part of a transaction that can be aborted here: https://github.com/apache/kafka/blob/trunk/connect/runtime/src/test/java/org/apache/kafka/connect/integration/MonitorableSourceConnector.java#L215-L216
   
   Because of this, the `awaitRecords` method is able to have the latch decremented enough number of times to be counted down to 0 within the timeout. 
   
   However, as you also rightly pointed out, the `recordToCommitLatch` is decremented for either cases of abort or commit of the txn which means `awaitCommits` also passes within the timeout.
   
   Could this make the test flaky since the bound on the `recordsRemainingLatch` doesn't seem to be strong enough to ensure that the actual number of records in the topic equals that? Would it help if we decrement the `recordsRemainingLatch` for cases of committed transactions only?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org